Nov 03 22:22:14.792656 ip-10-0-143-0 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 22:22:15.248996 ip-10-0-143-0 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:22:15.248996 ip-10-0-143-0 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 22:22:15.248996 ip-10-0-143-0 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:22:15.248996 ip-10-0-143-0 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:22:15.254696 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254617    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.254696 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254695    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.254696 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254698    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.254696 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254701    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254704    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254708    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254710    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254713    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254716    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254719    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254722    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254725    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254728    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254731    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254734    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254736    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254739    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254741    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254744    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254747    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254751    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254756    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.254814 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254759    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254761    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254764    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254767    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254770    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254772    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254775    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254777    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254780    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254782    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254785    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254787    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254790    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254792    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254795    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254798    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254801    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254805    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254807    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254811    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.255245 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254814    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254817    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254820    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254824    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254828    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254831    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254835    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254838    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254840    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254843    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254846    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254848    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254851    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254854    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254856    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254859    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254862    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254865    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254867    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.255734 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254870    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254872    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254875    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254878    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254880    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254883    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254886    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254889    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254893    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254895    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254898    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254901    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254904    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254906    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254909    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254911    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254914    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254916    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254919    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254921    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.256189 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254924    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254926    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254929    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254931    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254934    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254936    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254938    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254941    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254943    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254947    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254950    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254952    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254955    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254957    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254960    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254962    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254965    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254967    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254970    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.256736 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254973    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254976    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254978    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254981    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254983    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.254986    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255762    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255769    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255772    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255775    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255778    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255780    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255783    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255786    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255789    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255792    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255803    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255807    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255809    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255812    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.257314 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255815    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255817    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255820    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255823    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255825    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255828    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255831    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255834    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255836    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255839    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255842    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255845    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255847    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255850    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255853    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255856    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255859    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255861    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255865    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255868    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.257930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255870    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255873    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255876    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255878    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255881    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255884    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255886    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255889    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255891    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255894    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255897    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255899    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255902    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255905    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255907    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255910    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255913    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255916    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255918    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255921    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.258489 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255923    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255926    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255928    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255931    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255934    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255938    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255941    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255944    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255946    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255949    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255951    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255954    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255957    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255960    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255962    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255965    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255967    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255970    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255973    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255975    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.259062 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255978    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255980    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255984    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255988    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255991    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255994    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255996    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.255999    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256002    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256004    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256007    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256012    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256015    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256018    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256023    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256026    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256028    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256031    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.259619 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256034    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256037    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256039    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256042    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256044    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256047    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256050    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256053    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256055    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256057    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256060    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256062    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256065    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256067    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256561    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256568    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256570    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256574    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256576    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256579    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.260094 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256582    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256585    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256587    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256590    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256593    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256595    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256598    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256600    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256604    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256606    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256609    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256612    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256615    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256618    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256621    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256624    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256626    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256629    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256632    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256635    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.260623 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256638    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256641    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256643    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256646    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256648    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256651    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256653    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256656    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256658    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256661    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256664    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256666    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256669    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256671    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256674    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256676    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256679    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256681    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256684    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256686    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.261101 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256689    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256692    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256694    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256697    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256702    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256706    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256709    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256712    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256715    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256717    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256720    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256723    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256725    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256728    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256731    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256733    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256736    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256739    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256742    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.261582 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256745    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256747    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256750    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256753    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256755    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256758    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256760    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256763    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256765    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256768    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256770    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256773    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256775    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256778    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256781    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256783    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256786    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256789    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256791    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.262031 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256794    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256798    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256802    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256805    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256807    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256810    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256814    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256817    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256820    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256823    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256825    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256828    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256830    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256833    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256836    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256838    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256841    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256843    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256846    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.262493 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256849    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256851    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.256853    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.256956    2580 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.256973    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.256981    2580 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.256988    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.256995    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.256999    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257004    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257015    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257019    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257022    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257025    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257029    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257032    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257035    2580 flags.go:64] FLAG: --cgroup-root=""
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257038    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257041    2580 flags.go:64] FLAG: --client-ca-file=""
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257044    2580 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257048    2580 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257053    2580 flags.go:64] FLAG: --cluster-domain=""
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257056    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 22:22:15.262930 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257059    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257062    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257065    2580 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257069    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257072    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257075    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257078    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257082    2580 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257085    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257088    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257096    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257100    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257105    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257108    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257111    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257116    2580 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257119    2580 flags.go:64] FLAG: --enable-server="true"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257122    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257127    2580 flags.go:64] FLAG: --event-burst="100"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257131    2580 flags.go:64] FLAG: --event-qps="50"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257134    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257137    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257140    2580 flags.go:64] FLAG: --eviction-hard=""
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257145    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 22:22:15.263480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257147    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257151    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257154    2580 flags.go:64] FLAG: --eviction-soft=""
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257157    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257160    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257163    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257167    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257170    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257173    2580 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257176    2580 flags.go:64] FLAG: --feature-gates=""
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257180    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257182    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257186    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257189    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257192    2580 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257195    2580 flags.go:64] FLAG: --help="false"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257198    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257201    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257204    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257207    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257210    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257214    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257216    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 22:22:15.264595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257220    2580 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257223    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257226    2580 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257229    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257233    2580 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257236    2580 flags.go:64] FLAG: --kube-reserved=""
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257239    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257242    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257245    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257248    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257251    2580 flags.go:64] FLAG: --lock-file=""
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257254    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257257    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257260    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257266    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257269    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257273    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257276    2580 flags.go:64] FLAG: --logging-format="text"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257279    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257283    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257285    2580 flags.go:64] FLAG: --manifest-url=""
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257288    2580 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257293    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257296    2580 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257300    2580 flags.go:64] FLAG: --max-pods="110"
Nov 03 22:22:15.265581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257302    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257305    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257308    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257311    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257314    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257317    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257321    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257330    2580 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257333    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257336    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257339    2580 flags.go:64] FLAG: --pod-cidr=""
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257343    2580 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257346    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257349    2580 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257352    2580 flags.go:64] FLAG: --port="10250"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257355    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257358    2580 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0f0b51a50cc7d8862"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257361    2580 flags.go:64] FLAG: --qos-reserved=""
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257364    2580 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257367    2580 flags.go:64] FLAG: --register-node="true"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257370    2580 flags.go:64] FLAG: --register-with-taints=""
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257374    2580 flags.go:64] FLAG: --registry-burst="10"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257377    2580 flags.go:64] FLAG: --registry-qps="5"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257380    2580 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257383    2580 flags.go:64] FLAG: --reserved-memory=""
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257400    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 22:22:15.266370 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257403    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257406    2580 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257409    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257412    2580 flags.go:64] FLAG: --runonce="false"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257415    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257418    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257421    2580 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257424    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257427    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257430    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257433    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257436    2580 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257439    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257444    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257447    2580 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257450    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257453    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257458    2580 flags.go:64] FLAG: --system-cgroups=""
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257461    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257466    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257469    2580 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257472    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257477    2580 flags.go:64] FLAG: --tls-min-version=""
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257479    2580 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257482    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 22:22:15.267043 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257485    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257488    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257491    2580 flags.go:64] FLAG: --v="2"
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257496    2580 flags.go:64] FLAG: --version="false"
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257500    2580 flags.go:64] FLAG: --vmodule=""
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257504    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257508    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257652    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257656    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257660    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257663    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257666    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257669    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257672    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257675    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257678    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257680    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257683    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257686    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257688    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257692    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.267673 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257695    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257697    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257700    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257704    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257706    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257709    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257712    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257714    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257717    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257719    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257722    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257725    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257728    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257730    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257733    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257735    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257738    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257740    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257750    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257752    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.268182 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257755    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257758    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257761    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257763    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257766    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257768    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257771    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257773    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257776    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257778    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257781    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257785    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257788    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257790    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257793    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257798    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257801    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257804    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257806    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.268675 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257809    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257812    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257814    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257817    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257819    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257822    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257825    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257827    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257830    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257832    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257835    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257837    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257840    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257844    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257846    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257849    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257851    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257854    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257856    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257859    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.269146 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257861    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257865    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257869    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257873    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257878    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257882    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257884    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257887    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257891    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257893    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257896    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257899    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257902    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257904    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257907    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257909    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257912    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257914    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257917    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.269634 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257920    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257922    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257925    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257927    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257930    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257933    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257936    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257939    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257941    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257944    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257947    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257949    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257952    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.257954    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.257982    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258021    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.270083 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258025    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258030    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258033    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258035    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258038    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258041    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258045    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258048    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258051    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258053    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258056    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258058    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258061    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258064    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258067    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258069    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258072    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258074    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258077    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258080    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.270501 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258083    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258085    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258088    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258091    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258093    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258096    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258098    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258101    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258103    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258106    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258109    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258111    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258114    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258117    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258120    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258123    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258125    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258128    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258132    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.270962 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258135    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258138    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258140    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258143    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258145    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258148    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258151    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258153    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258155    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258158    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258161    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258163    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258166    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258169    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258171    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258174    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258176    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258179    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258182    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258184    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.271419 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258187    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258192    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258195    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258199    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258202    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258205    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258209    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258212    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258214    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258217    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258220    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258224    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258227    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258230    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258233    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258235    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258238    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258240    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258243    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.271898 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258246    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258248    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258251    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258253    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258256    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258259    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258261    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258264    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258266    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258269    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258271    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258274    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258277    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258280    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258282    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258285    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258287    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258290    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258293    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258296    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.272333 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258299    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258303    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258305    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258307    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258311    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258314    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.258316    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.265940    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.265961    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.266729    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.266746    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266807    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266812    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266816    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266820    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266823    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266826    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266829    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266832    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266835    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.272823 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266837    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266840    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266843    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266846    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266848    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266851    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266853    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266856    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266859    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266861    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266864    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266866    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266869    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266871    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266874    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266876    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266879    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266881    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266884    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266887    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.273289 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266891    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266894    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266897    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266900    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266903    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266905    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266909    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266913    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266916    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266919    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266922    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266924    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266927    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266929    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266932    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266935    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266937    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266940    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266943    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.273772 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266945    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266948    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266950    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266953    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266956    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266960    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266964    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266967    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266969    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266972    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266975    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266977    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266980    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266983    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266986    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266989    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266992    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266995    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.266997    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.274201 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267000    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267002    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267005    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267008    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267010    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267013    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267015    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267018    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267020    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267023    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267025    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267027    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267030    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267033    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267035    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267038    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267040    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267043    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267046    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267048    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.274767 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267051    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267053    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267056    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267059    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267062    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267065    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267067    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267070    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267072    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267075    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267078    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267080    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267083    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267085    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267088    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267090    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267093    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267095    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.275283 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267098    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.267126    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267260    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267265    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267268    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267271    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267273    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267276    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267279    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267281    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267284    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267287    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267289    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267292    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267294    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267297    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.275733 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267299    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267303    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267307    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267311    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267313    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267316    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267319    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267322    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267324    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267327    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267330    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267332    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267335    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267337    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267340    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267342    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267345    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267347    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.276350 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267350    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267352    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267355    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267358    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267361    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267364    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267366    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267369    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267371    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267374    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267376    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267379    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267381    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267384    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267402    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267406    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267409    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267412    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267415    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267417    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.276957 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267420    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267423    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267426    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267428    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267431    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267434    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267436    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267439    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267441    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267444    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267446    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267449    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267451    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267454    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267457    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267459    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267462    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267464    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267466    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.277524 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267469    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267472    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267474    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267476    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267479    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267481    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267484    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267486    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267489    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267491    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267494    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267497    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267499    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267502    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267504    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267507    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267511    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267514    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267516    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267519    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.278092 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267521    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267524    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267526    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267529    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267531    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267534    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267536    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267539    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267541    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267544    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267546    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267549    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267551    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267554    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267556    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.267580    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:22:15.278744 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267610    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267614    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267616    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267619    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267622    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267627    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267629    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267632    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267635    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267638    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267640    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267643    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267646    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267648    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267651    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267653    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267656    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267658    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267661    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:15.279122 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267663    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267666    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267668    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267671    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267673    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267676    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267678    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267681    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267684    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267686    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267689    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267691    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267694    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267696    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267699    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267701    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267704    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267706    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267710    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267713    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:15.279575 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267717    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267720    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267722    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267725    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267728    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267730    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267733    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267735    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267738    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267741    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267744    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267746    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267749    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267751    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267754    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267756    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267759    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267762    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267764    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267767    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:15.280058 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267769    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267772    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267774    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267777    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267780    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267782    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267785    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267787    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267790    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267793    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267806    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267811    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267816    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267819    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267822    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267825    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267828    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267830    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267833    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:15.280560 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267835    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267843    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267846    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267849    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267852    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267854    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267857    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267859    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267862    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267864    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267867    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267871    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267875    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267877    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267880    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267882    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267885    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267887    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267890    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267893    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:15.281001 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267895    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267898    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267900    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267904    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267906    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267911    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267914    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:15.267916    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.267967    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.270206    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.271247    2580 server.go:1014] "Starting client certificate rotation"
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.271344    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.271382    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.274226    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:22:15.281467 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.280585    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:22:15.292694 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.292678    2580 log.go:25] "Validated CRI v1 runtime API"
Nov 03 22:22:15.295944 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.295929    2580 log.go:25] "Validated CRI v1 image API"
Nov 03 22:22:15.296773 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.296751    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 22:22:15.301115 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.301097    2580 fs.go:135] Filesystem UUIDs: map[18a61e18-3240-49cd-845c-59f808783e31:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b567883c-693a-4e55-84dc-394080a948b1:/dev/nvme0n1p4]
Nov 03 22:22:15.301191 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.301114    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 03 22:22:15.306876 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.306763    2580 manager.go:217] Machine: {Timestamp:2026-11-03 22:22:15.305046994 +0000 UTC m=+0.384503922 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106408 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f9b28548d20a661f9a9fb7ca277cc SystemUUID:ec2f9b28-548d-20a6-61f9-a9fb7ca277cc BootID:755477c3-708b-4e1f-8200-8060d5b7d01e Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:df:06:ec:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:df:06:ec:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:ed:cc:e4:3d:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 22:22:15.307412 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.307402    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 22:22:15.307496 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.307485    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 22:22:15.310480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.310454    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 22:22:15.310604 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.310483    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-0.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 22:22:15.310645 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.310606    2580 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 22:22:15.310645 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.310614    2580 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 22:22:15.310645 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.310636    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:22:15.311306 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.311296    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:22:15.312349 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.312339    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 22:22:15.316632 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.316621    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 22:22:15.316742 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.316733    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 22:22:15.317433 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.317423    2580 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 22:22:15.317470 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.317437    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 22:22:15.317470 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.317448    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 22:22:15.317470 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.317457    2580 kubelet.go:405] "Adding apiserver pod source"
Nov 03 22:22:15.317470 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.317465    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 22:22:15.317791 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.317777    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:22:15.320439 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.320421    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 22:22:15.326515 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.326502    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:22:15.327673 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327661    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 22:22:15.327708 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327691    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 22:22:15.327865 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327848    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 22:22:15.327896 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327871    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 22:22:15.327896 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327879    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 22:22:15.327896 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327888    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 22:22:15.327896 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327896    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 22:22:15.327995 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.327903    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 22:22:15.328429 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.328420    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 22:22:15.328464 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.328430    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 22:22:15.328464 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.328437    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 22:22:15.328464 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.328443    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 22:22:15.328464 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.328459    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 22:22:15.328663 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.328468    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 22:22:15.330160 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.330151    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 22:22:15.330200 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.330163    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 22:22:15.333858 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.333841    2580 server.go:1263] "Started kubelet"
Nov 03 22:22:15.334816 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.334779    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 22:22:15.335802 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.335746    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 22:22:15.335882 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.335808    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 22:22:15.336676 ip-10-0-143-0 systemd[1]: Started Kubernetes Kubelet.
Nov 03 22:22:15.338010 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.337995    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 22:22:15.340135 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.340119    2580 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 22:22:15.343357 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.343338    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 22:22:15.343447 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.343356    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 22:22:15.343542 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.343526    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 03 22:22:15.343919 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.343885    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:15.344531 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.344512    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 22:22:15.344531 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.344522    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 22:22:15.344677 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.344537    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 22:22:15.344677 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.344659    2580 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 22:22:15.344677 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.344666    2580 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 22:22:15.345483 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.345467    2580 factory.go:55] Registering systemd factory
Nov 03 22:22:15.345483 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.345482    2580 factory.go:223] Registration of the systemd container factory successfully
Nov 03 22:22:15.345776 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.345762    2580 factory.go:153] Registering CRI-O factory
Nov 03 22:22:15.345825 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.345779    2580 factory.go:223] Registration of the crio container factory successfully
Nov 03 22:22:15.345854 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.345826    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 22:22:15.345854 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.345844    2580 factory.go:103] Registering Raw factory
Nov 03 22:22:15.345932 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.345857    2580 manager.go:1199] Started watching for new ooms in manager
Nov 03 22:22:15.346791 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.346768    2580 manager.go:319] Starting recovery of all containers
Nov 03 22:22:15.355848 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.355821    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 22:22:15.356216 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.356198    2580 manager.go:324] Recovery completed
Nov 03 22:22:15.361070 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.361052    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.362891 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.362877    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.362954 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.362905    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.362954 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.362922    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.363437 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.363422    2580 cpu_manager.go:226] "Starting" policy="none"
Nov 03 22:22:15.363437 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.363435    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 22:22:15.363560 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.363454    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 22:22:15.363560 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.363464    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.364068 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.364055    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.364119 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.364077    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.364119 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.364086    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.365897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.365882    2580 policy_none.go:50] "Start"
Nov 03 22:22:15.365974 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.365901    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 22:22:15.365974 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.365914    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 22:22:15.367932 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.367919    2580 policy_none.go:44] "Start"
Nov 03 22:22:15.422006 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.421975    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 22:22:15.424992 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.424971    2580 manager.go:341] "Starting Device Plugin manager"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.425013    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.425023    2580 server.go:87] "Starting device plugin registration server"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.425295    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.425309    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.425770    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.425867    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.425874    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.426164    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 22:22:15.426665 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.426217    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:15.476263 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.476244    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:15.478140 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.476277    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 03 22:22:15.492018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.491995    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 22:22:15.492018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.492020    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 22:22:15.492132 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.492038    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 22:22:15.492132 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.492067    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 03 22:22:15.522925 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.521913    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b65ec21c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.333814812 +0000 UTC m=+0.413271704,LastTimestamp:2026-11-03 22:22:15.333814812 +0000 UTC m=+0.413271704,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:15.525574 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.525561    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.526287 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.526273    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.526337 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.526300    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.526337 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.526311    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.526337 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.526333    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.528685 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.528664    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 03 22:22:15.528772 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.528755    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 22:22:15.593515 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.593492    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"]
Nov 03 22:22:15.593595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.593545    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.594496 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.594480    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.594554 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.594509    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.594554 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.594520    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.596534 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.596522    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.596668 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.596652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.596715 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.596687    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.597305 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.597288    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.597379 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.597317    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.597379 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.597294    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.597379 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.597346    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.597379 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.597358    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.597379 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.597328    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.599376 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.599360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.599463 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.599385    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.600288 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.600272    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.600367 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.600303    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.600367 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.600319    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.624483 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.624386    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:15.624584 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.624465    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 22:22:15.626593 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.626575    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.629709 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.629694    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.646463 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.646443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.646529 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.646466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ffed3ef5033ca77208390bb2ae5eac19-config\") pod \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"ffed3ef5033ca77208390bb2ae5eac19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.646529 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.646484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.693411 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.693367    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.693493 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.693407    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 03 22:22:15.693532 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.693438    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,LastTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:15.747515 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.747495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ffed3ef5033ca77208390bb2ae5eac19-config\") pod \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"ffed3ef5033ca77208390bb2ae5eac19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.747607 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.747522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.747607 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.747540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.747607 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.747601    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.747702 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.747608    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.747702 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.747601    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ffed3ef5033ca77208390bb2ae5eac19-config\") pod \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"ffed3ef5033ca77208390bb2ae5eac19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.786814 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.786771    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Nov 03 22:22:15.786875 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.786811    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,LastTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:15.841793 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.841724    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.364066841 +0000 UTC m=+0.443523733,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:15.894706 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.894681    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:15.895668 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.895652    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:15.895748 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.895683    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:15.895748 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.895714    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:15.895748 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.895748    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.930818 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.930796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.934297 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:15.934270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:15.972759 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:15.972508    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,LastTimestamp:2026-11-03 22:22:15.364081851 +0000 UTC m=+0.443538743,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.057581 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.057524    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:16.057662 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.057526    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,LastTimestamp:2026-11-03 22:22:15.364090246 +0000 UTC m=+0.443547141,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.137730 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.137653    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926bbcd16de  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated limits on system reserved cgroup /system.slice,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.42493155 +0000 UTC m=+0.504388449,LastTimestamp:2026-11-03 22:22:15.42493155 +0000 UTC m=+0.504388449,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.239634 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.239550    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926bbf72497  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.427687575 +0000 UTC m=+0.507144468,LastTimestamp:2026-11-03 22:22:15.427687575 +0000 UTC m=+0.507144468,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.298809 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.298777    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Nov 03 22:22:16.299251 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.298778    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.526287675 +0000 UTC m=+0.605744567,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.407989 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.407855    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,LastTimestamp:2026-11-03 22:22:15.526305102 +0000 UTC m=+0.605761995,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.457752 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.457729    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:16.458703 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.458684    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:16.458773 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.458717    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:16.458773 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.458729    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:16.458773 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.458754    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:16.477352 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.477282    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,LastTimestamp:2026-11-03 22:22:15.526314711 +0000 UTC m=+0.605771603,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.568816 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.568736    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.594494752 +0000 UTC m=+0.673951643,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.568967 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.568853    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:16.626707 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.626681    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:16.627739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.627719    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:16.627839 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.627759    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:16.627839 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:16.627775    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:16.656399 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.656370    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:16.656577 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.656490    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,LastTimestamp:2026-11-03 22:22:15.59451409 +0000 UTC m=+0.673970982,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.743645 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.743491    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,LastTimestamp:2026-11-03 22:22:15.594523801 +0000 UTC m=+0.673980692,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.834978 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.834886    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.597307161 +0000 UTC m=+0.676764056,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.835166 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.835140    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 22:22:16.917334 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.917301    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 03 22:22:16.917473 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.917289    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,LastTimestamp:2026-11-03 22:22:15.59732245 +0000 UTC m=+0.676779342,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:16.995537 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.995474    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 22:22:16.995537 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:16.995440    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.597334273 +0000 UTC m=+0.676791165,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.082218 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.082114    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,LastTimestamp:2026-11-03 22:22:15.597352491 +0000 UTC m=+0.676809383,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.159229 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:17.159185    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c3a967cbec18dc888ae76406901416.slice/crio-fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957 WatchSource:0}: Error finding container fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957: Status 404 returned error can't find the container with id fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957
Nov 03 22:22:17.159546 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:17.159522    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffed3ef5033ca77208390bb2ae5eac19.slice/crio-80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c WatchSource:0}: Error finding container 80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c: Status 404 returned error can't find the container with id 80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c
Nov 03 22:22:17.164207 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.164191    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:22:17.172008 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.171982    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Nov 03 22:22:17.172151 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.172039    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,LastTimestamp:2026-11-03 22:22:15.597362817 +0000 UTC m=+0.676819713,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.256952 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.256809    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,LastTimestamp:2026-11-03 22:22:15.597378059 +0000 UTC m=+0.676834954,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.333010 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.332973    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 03 22:22:17.333368 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.333018    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.60028655 +0000 UTC m=+0.679743447,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.422461 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.422359    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81abf66  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362912102 +0000 UTC m=+0.442368998,LastTimestamp:2026-11-03 22:22:15.600310605 +0000 UTC m=+0.679767498,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.456751 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.456729    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:17.457803 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.457783    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:17.457897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.457821    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:17.457897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.457837    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:17.457897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.457874    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:17.496688 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.496643    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerStarted","Data":"fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957"}
Nov 03 22:22:17.497717 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.497699    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"ffed3ef5033ca77208390bb2ae5eac19","Type":"ContainerStarted","Data":"80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c"}
Nov 03 22:22:17.508140 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.508035    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81afa6b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362927211 +0000 UTC m=+0.442384102,LastTimestamp:2026-11-03 22:22:15.600324199 +0000 UTC m=+0.679781093,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.595173 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:17.595150    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:17.595318 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.595207    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-0.us-west-2.compute.internal.18e4a926b81a6ff2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-0.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-0.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:15.362891762 +0000 UTC m=+0.442348654,LastTimestamp:2026-11-03 22:22:15.895667245 +0000 UTC m=+0.975124140,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.689400 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.689353    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:17.689532 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.689424    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal.18e4a92723894d63  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal,UID:ffed3ef5033ca77208390bb2ae5eac19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e\",Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:17.165319523 +0000 UTC m=+2.244776407,LastTimestamp:2026-11-03 22:22:17.165319523 +0000 UTC m=+2.244776407,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:17.755242 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:17.755162    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal.18e4a92723a4dc08  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal,UID:43c3a967cbec18dc888ae76406901416,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d017b7d199b1d54e889bb1eb6e572d8e6d94ad92610894009ee0562c2a5c0f51\",Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:17.167125512 +0000 UTC m=+2.246582396,LastTimestamp:2026-11-03 22:22:17.167125512 +0000 UTC m=+2.246582396,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:18.533778 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:18.533743    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:18.848889 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:18.848792    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 03 22:22:18.849026 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:18.848879    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Nov 03 22:22:18.849026 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:18.848993    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 22:22:18.938940 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:18.938908    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:18.940692 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:18.940120    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:18.940692 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:18.940160    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:18.940692 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:18.940177    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:18.963657 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:18.963625    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 22:22:19.290529 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:19.290331    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:19.291480 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:19.291458    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:19.291580 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:19.291503    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:19.291580 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:19.291521    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:19.291580 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:19.291562    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:19.454410 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:19.454280    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal.18e4a927a259f80b  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal,UID:43c3a967cbec18dc888ae76406901416,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d017b7d199b1d54e889bb1eb6e572d8e6d94ad92610894009ee0562c2a5c0f51\" in 2.125s (2.125s including waiting). Image size: 272339155 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:19.292923915 +0000 UTC m=+4.372380814,LastTimestamp:2026-11-03 22:22:19.292923915 +0000 UTC m=+4.372380814,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:19.635789 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:19.635762    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:19.810689 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:19.810621    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:19.810689 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:19.810630    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal.18e4a927b570922c  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal,UID:43c3a967cbec18dc888ae76406901416,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Container created,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:19.613172268 +0000 UTC m=+4.692629173,LastTimestamp:2026-11-03 22:22:19.613172268 +0000 UTC m=+4.692629173,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:19.975667 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:19.975546    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal.18e4a927b5f9c047  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal,UID:43c3a967cbec18dc888ae76406901416,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Container started,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:19.622162503 +0000 UTC m=+4.701619400,LastTimestamp:2026-11-03 22:22:19.622162503 +0000 UTC m=+4.701619400,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:20.158361 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:20.158295    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 03 22:22:20.158361 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:20.158260    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal.18e4a927b8ead7db  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal,UID:ffed3ef5033ca77208390bb2ae5eac19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e\" in 2.506s (2.506s including waiting). Image size: 294084393 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:19.671517147 +0000 UTC m=+4.750974052,LastTimestamp:2026-11-03 22:22:19.671517147 +0000 UTC m=+4.750974052,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:20.337363 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:20.337266    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal.18e4a927bb8afa05  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal,UID:ffed3ef5033ca77208390bb2ae5eac19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Container created,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:19.715566085 +0000 UTC m=+4.795023001,LastTimestamp:2026-11-03 22:22:19.715566085 +0000 UTC m=+4.795023001,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:20.505275 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.505178    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerStarted","Data":"3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a"}
Nov 03 22:22:20.505275 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.505225    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:20.506312 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.506292    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:20.506423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.506330    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:20.506423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.506357    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:20.506584 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:20.506567    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:20.506838 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.506814    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"ffed3ef5033ca77208390bb2ae5eac19","Type":"ContainerStarted","Data":"3b3943e51c8daa17cf549fffd584148f5f90ab86cf8c4be72170ac1425b41055"}
Nov 03 22:22:20.506885 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.506849    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:20.507569 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.507555    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:20.507615 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.507583    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:20.507615 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.507598    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:20.507746 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:20.507734    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:20.516637 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:20.516555    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal.18e4a927bbfd1409  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal,UID:ffed3ef5033ca77208390bb2ae5eac19,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Container started,Source:EventSource{Component:kubelet,Host:ip-10-0-143-0.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 22:22:19.723043849 +0000 UTC m=+4.802500731,LastTimestamp:2026-11-03 22:22:19.723043849 +0000 UTC m=+4.802500731,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-0.us-west-2.compute.internal,}"
Nov 03 22:22:20.680072 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:20.680043    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:21.508812 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.508779    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:21.508981 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.508784    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:21.509619 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.509599    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:21.509715 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.509628    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:21.509715 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.509640    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:21.509715 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.509700    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:21.509821 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.509729    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:21.509821 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.509744    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:21.509899 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:21.509848    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:21.509968 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:21.509951    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:21.653385 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:21.653361    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:22.209931 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:22.209899    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Nov 03 22:22:22.702302 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:22.702219    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:22.702468 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:22.702330    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 03 22:22:23.015290 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.015189    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:23.016429 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.016392    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:23.016539 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.016443    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:23.016539 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.016456    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:23.016539 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.016489    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:23.346370 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:23.346288    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:23.656551 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.656476    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:23.790805 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.790762    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:23.791910 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.791890    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:23.792013 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.791925    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:23.792013 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:23.791936    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:24.331798 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:24.331765    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-0.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 22:22:24.627070 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:24.626987    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:24.793019 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:24.792986    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 03 22:22:25.427188 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:25.427150    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:25.547150 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:25.547122    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-0.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:22:25.547311 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:25.547223    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 22:22:25.572703 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:25.572680    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 22:22:25.572774 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:25.572734    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-whc8k"
Nov 03 22:22:26.271364 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:26.271317    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 03 22:22:26.576428 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:26.576331    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 22:17:25 +0000 UTC" deadline="2028-08-21 10:23:48.528967866 +0000 UTC"
Nov 03 22:22:26.576428 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:26.576364    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15756h1m21.952607477s"
Nov 03 22:22:26.712614 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:26.712587    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:26.781105 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:26.781080    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:26.893118 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:26.893070    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:27.206973 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:27.206913    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:27.206973 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:27.206943    2580 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:27.296729 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:27.296706    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:27.365523 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:27.365500    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:27.478630 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:27.478580    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:27.808016 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:27.807941    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:27.808016 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:27.807967    2580 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:28.119212 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:28.119143    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:28.183339 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:28.183318    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:28.289643 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:28.289622    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:28.613621 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:28.613556    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:28.613621 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:28.613581    2580 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-143-0.us-west-2.compute.internal" not found
Nov 03 22:22:28.717660 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:28.717640    2580 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:29.746533 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:29.746496    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:29.750963 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:29.750945    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:29.751067 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:29.750976    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:29.751067 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:29.750988    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:29.751067 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:29.751017    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:29.812640 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:29.812620    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:29.812690 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:29.812644    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-0.us-west-2.compute.internal\": node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:29.929222 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:29.929196    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.029665 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.029616    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.130150 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.130124    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.230615 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.230596    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.331722 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.331671    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.431866 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.431847    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.532316 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.532295    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.633116 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.633065    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.734089 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.733860    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.834533 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.834502    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:30.934989 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:30.934922    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.035598 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.035573    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.054374 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.054355    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:31.136541 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.136509    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.237005 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.236944    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.337867 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.337829    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.438448 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.438415    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.528073 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.528020    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal_43c3a967cbec18dc888ae76406901416/setup/0.log"
Nov 03 22:22:31.528073 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.528053    2580 generic.go:356] "Generic (PLEG): container finished" podID="43c3a967cbec18dc888ae76406901416" containerID="3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a" exitCode=1
Nov 03 22:22:31.528197 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.528083    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerDied","Data":"3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a"}
Nov 03 22:22:31.528229 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.528207    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:31.529153 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.529137    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:31.529203 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.529165    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:31.529203 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:31.529175    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:31.529339 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.529327    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:31.538707 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.538679    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.639217 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.639069    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.740287 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.740265    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.841064 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.841014    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:31.941543 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:31.941512    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.042162 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.042136    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.090742 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:32.090722    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:32.143031 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.142967    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.243540 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.243507    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.344330 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.344308    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.444542 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.444490    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.531742 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:32.531722    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal_43c3a967cbec18dc888ae76406901416/setup/0.log"
Nov 03 22:22:32.531810 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:32.531762    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerStarted","Data":"97dbe1ff7ccbcc404d29d98640c63290b94ec8634808b2f5b49138b10aaeafff"}
Nov 03 22:22:32.531872 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:32.531861    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:32.532816 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:32.532797    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:32.532897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:32.532829    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:32.532897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:32.532840    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:32.533007 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.532995    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:32.545169 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.545146    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.645486 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.645455    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.745872 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.745817    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.846476 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.846454    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:32.946924 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:32.946899    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.047969 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.047898    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.148911 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.148877    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.249356 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.249317    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.277104 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:33.277087    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:33.278130 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:33.278095    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:33.278223 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:33.278137    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:33.278223 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:33.278147    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:33.349648 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.349592    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.450222 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.450188    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.551184 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.551162    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.651671 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.651613    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.752126 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.752087    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.852764 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.852734    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:33.953214 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:33.953161    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.053614 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.053586    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.154108 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.154073    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.255086 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.255018    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.355937 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.355898    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.456780 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.456755    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.557786 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.557732    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.658585 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.658551    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.759011 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.758979    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.859788 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.859730    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:34.960209 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:34.960175    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:35.060695 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:35.060654    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:35.161133 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:35.161079    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:35.262140 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:35.262107    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:35.362739 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:35.362704    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:22:35.379177 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:35.379159    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:35.444706 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:35.444641    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:35.506880 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:35.506862    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:22:35.509315 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:35.509302    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:22:35.568595 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:35.568584    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:22:36.331251 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.331206    2580 apiserver.go:52] "Watching apiserver"
Nov 03 22:22:36.358535 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.358513    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 03 22:22:36.582927 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.582843    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 22:22:36.583204 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.583185    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j","openshift-multus/network-metrics-daemon-glv5p","kube-system/global-pull-secret-syncer-86xxh","kube-system/konnectivity-agent-d46vm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal","openshift-network-diagnostics/network-check-target-cvzp9","openshift-cluster-node-tuning-operator/tuned-5rr7s","openshift-dns/node-resolver-vr4x9","openshift-multus/multus-additional-cni-plugins-qj4mh","openshift-ovn-kubernetes/ovnkube-node-lskjf","kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal","openshift-image-registry/node-ca-5dtsg","openshift-multus/multus-6nlml","openshift-network-operator/iptables-alerter-kzxsv"]
Nov 03 22:22:36.585482 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.585462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.587404 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.587370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.589248 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.589231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.589323 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.589295    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:36.591155 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.591137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.593136 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.593117    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.594933 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.594916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:36.595021 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.594969    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:36.596967 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.596948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.598943 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.598926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.601768 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.601753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:36.601837 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.601813    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:36.603835 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.603816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.605862 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.605848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.608433 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.608414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.622700 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.622681    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 22:22:36.622779 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.622772    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8k4gz"
Nov 03 22:22:36.622839 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.622783    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8k4gz"
Nov 03 22:22:36.640259 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.640241    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hvj2k\""
Nov 03 22:22:36.640778 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.640762    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nh9f8\""
Nov 03 22:22:36.645494 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.645477    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 22:22:36.677031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.677005    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mdpv5\""
Nov 03 22:22:36.677352 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.677336    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 22:22:36.677447 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.677434    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 22:22:36.677548 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.677535    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g6sdx\""
Nov 03 22:22:36.677819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.677635    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-ljhlr\""
Nov 03 22:22:36.679229 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.677914    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ztkgs\""
Nov 03 22:22:36.722273 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722253    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.722374 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722332    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.722478 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722463    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 22:22:36.722571 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722551    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.722647 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722579    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.722647 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722629    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 22:22:36.722647 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722635    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.722647 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722637    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.722779 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722647    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 22:22:36.722779 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722655    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.722779 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722705    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 22:22:36.722779 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722748    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 22:22:36.722779 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722772    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722801    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722827    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722827    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722844    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722851    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722889    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722907    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722908    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722912    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.722977 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.722949    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.723268 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.723056    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 22:22:36.725213 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-run\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.725258 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-kubelet\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.725258 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.725338 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-multus\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.725338 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-host\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.725436 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s522\" (UniqueName: \"kubernetes.io/projected/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-kube-api-access-7s522\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.725436 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51712151-3879-46f0-99fb-919861bf9ee5-tmp-dir\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.725532 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-socket-dir-parent\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.725532 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-netns\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.725532 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5hh9\" (UniqueName: \"kubernetes.io/projected/e7561551-7ec6-43f6-9542-77c479bd3975-kube-api-access-q5hh9\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.725650 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-systemd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.725650 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-etc-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.725650 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-log-socket\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.725650 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-netd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.725796 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725669    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.725796 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vxhc\" (UniqueName: \"kubernetes.io/projected/51712151-3879-46f0-99fb-919861bf9ee5-kube-api-access-4vxhc\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.725796 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:36.725796 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.725796 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725810    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfj66\" (UniqueName: \"kubernetes.io/projected/2ebc48f1-c278-4176-9256-5e017ca78b2c-kube-api-access-xfj66\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-ovn\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-multus-daemon-config\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-iptables-alerter-script\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-tuned\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-agent-certs\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.725999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-system-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.726018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-netns\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phrvx\" (UniqueName: \"kubernetes.io/projected/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-kube-api-access-phrvx\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysconfig\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-sys-fs\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff5c6570-b247-497c-aa51-90e92c38db3c-host\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-bin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-var-lib-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-conf\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff5c6570-b247-497c-aa51-90e92c38db3c-serviceca\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cnibin\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-kubelet\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-systemd-units\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98skz\" (UniqueName: \"kubernetes.io/projected/f97dec7e-9808-4b45-8007-fbaacb1f65b5-kube-api-access-98skz\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.726423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-kubelet-config\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-kubernetes\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-lib-modules\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-tmp\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-multus-certs\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fwjp\" (UniqueName: \"kubernetes.io/projected/ff5c6570-b247-497c-aa51-90e92c38db3c-kube-api-access-8fwjp\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51712151-3879-46f0-99fb-919861bf9ee5-hosts-file\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-socket-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqfx7\" (UniqueName: \"kubernetes.io/projected/fa1fb598-13ff-4b53-bec5-85cf10855d09-kube-api-access-fqfx7\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-k8s-cni-cncf-io\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-script-lib\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-dbus\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-cnibin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-hostroot\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.727031 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-conf-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-config\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-host-slash\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-sys\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-node-log\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-systemd\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.726983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-var-lib-kubelet\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-konnectivity-ca\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-registration-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-device-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-system-cni-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-slash\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-cni-binary-copy\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-env-overrides\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.727667 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727250    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.728072 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-etc-kubernetes\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.728072 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-bin\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.728072 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-modprobe-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.728072 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.727374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brhp9\" (UniqueName: \"kubernetes.io/projected/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-kube-api-access-brhp9\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.741452 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.741428    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kfchj\""
Nov 03 22:22:36.741452 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.741433    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 22:22:36.741452 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.741433    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kccnl\""
Nov 03 22:22:36.741588 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.741456    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hp7n8\""
Nov 03 22:22:36.827937 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.827913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51712151-3879-46f0-99fb-919861bf9ee5-tmp-dir\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.827937 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.827938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-socket-dir-parent\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.827953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-netns\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.827968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5hh9\" (UniqueName: \"kubernetes.io/projected/e7561551-7ec6-43f6-9542-77c479bd3975-kube-api-access-q5hh9\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.827982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-systemd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.827997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-etc-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-log-socket\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-netd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828043    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-socket-dir-parent\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828059    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-netns\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828087    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-etc-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828098    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-netd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vxhc\" (UniqueName: \"kubernetes.io/projected/51712151-3879-46f0-99fb-919861bf9ee5-kube-api-access-4vxhc\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828132    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-log-socket\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828137    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-systemd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfj66\" (UniqueName: \"kubernetes.io/projected/2ebc48f1-c278-4176-9256-5e017ca78b2c-kube-api-access-xfj66\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-ovn\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828312    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51712151-3879-46f0-99fb-919861bf9ee5-tmp-dir\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828321    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-ovn\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-multus-daemon-config\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828239    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.828316    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-iptables-alerter-script\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-tuned\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.828596 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.828437    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs podName:2ebc48f1-c278-4176-9256-5e017ca78b2c nodeName:}" failed. No retries permitted until 2026-11-03 22:22:37.328415367 +0000 UTC m=+22.407872255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs") pod "network-metrics-daemon-glv5p" (UID: "2ebc48f1-c278-4176-9256-5e017ca78b2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-agent-certs\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-system-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-netns\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phrvx\" (UniqueName: \"kubernetes.io/projected/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-kube-api-access-phrvx\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysconfig\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828633    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-system-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-sys-fs\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828671    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-netns\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff5c6570-b247-497c-aa51-90e92c38db3c-host\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-bin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-var-lib-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-conf\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828750    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysconfig\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff5c6570-b247-497c-aa51-90e92c38db3c-serviceca\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828771    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828798    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff5c6570-b247-497c-aa51-90e92c38db3c-host\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.829147 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828806    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-var-lib-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cnibin\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828836    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-bin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-kubelet\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-systemd-units\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828880    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-sys-fs\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98skz\" (UniqueName: \"kubernetes.io/projected/f97dec7e-9808-4b45-8007-fbaacb1f65b5-kube-api-access-98skz\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-systemd-units\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-kubelet-config\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828957    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cnibin\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-kubernetes\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-lib-modules\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-tmp\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.830012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-multus-certs\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829035    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fwjp\" (UniqueName: \"kubernetes.io/projected/ff5c6570-b247-497c-aa51-90e92c38db3c-kube-api-access-8fwjp\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829000    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-conf\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51712151-3879-46f0-99fb-919861bf9ee5-hosts-file\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829085    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-kubernetes\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829115    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51712151-3879-46f0-99fb-919861bf9ee5-hosts-file\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829034    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-kubelet-config\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829160    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-multus-certs\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829212    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-lib-modules\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829223    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.828923    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-kubelet\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-socket-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqfx7\" (UniqueName: \"kubernetes.io/projected/fa1fb598-13ff-4b53-bec5-85cf10855d09-kube-api-access-fqfx7\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-k8s-cni-cncf-io\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-script-lib\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.830819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-dbus\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-cnibin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829452    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-k8s-cni-cncf-io\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-hostroot\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-conf-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829493    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829547    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-cnibin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829556    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-socket-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829567    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-dbus\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829602    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-hostroot\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-config\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-host-slash\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-sys\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-node-log\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-systemd\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.831552 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-var-lib-kubelet\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829776    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-sys\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829785    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-node-log\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-konnectivity-ca\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829832    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-registration-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.829849    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-device-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829883    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-registration-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.829895    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret podName:48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:37.329879262 +0000 UTC m=+22.409336172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret") pod "global-pull-secret-syncer-86xxh" (UID: "48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829905    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-device-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829912    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-host-slash\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829944    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-systemd\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829954    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-var-lib-kubelet\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-system-cni-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829990    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-conf-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.832417 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.829997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-slash\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830030    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-slash\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830028    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-system-cni-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-cni-binary-copy\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-env-overrides\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-etc-kubernetes\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-bin\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-modprobe-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brhp9\" (UniqueName: \"kubernetes.io/projected/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-kube-api-access-brhp9\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-run\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-kubelet\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830258    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-bin\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-multus\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-host\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s522\" (UniqueName: \"kubernetes.io/projected/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-kube-api-access-7s522\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.833189 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830296    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830440    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-etc-kubernetes\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830526    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-modprobe-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830756    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-multus\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830783    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-run\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-host\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.830834    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-kubelet\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.831109    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.831183    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-multus-daemon-config\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.831261    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-iptables-alerter-script\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.832177    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff5c6570-b247-497c-aa51-90e92c38db3c-serviceca\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.832298    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-cni-binary-copy\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.832318    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-env-overrides\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.832411    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.832702    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-config\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.832942    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-script-lib\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.833068    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-konnectivity-ca\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.833073    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.833739 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.833454    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-tmp\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.834211 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.833460    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.834211 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.833527    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-tuned\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.834211 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.833688    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-agent-certs\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.888840 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.888812    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:36.888840 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.888841    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:36.888971 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.888854    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vp9qw for pod openshift-network-diagnostics/network-check-target-cvzp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:36.888971 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:36.888912    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw podName:571f0155-36fa-4eb3-9ddd-9317472e6761 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:37.388900179 +0000 UTC m=+22.468357093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vp9qw" (UniqueName: "kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw") pod "network-check-target-cvzp9" (UID: "571f0155-36fa-4eb3-9ddd-9317472e6761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:36.890897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.890762    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vxhc\" (UniqueName: \"kubernetes.io/projected/51712151-3879-46f0-99fb-919861bf9ee5-kube-api-access-4vxhc\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.890897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.890762    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5hh9\" (UniqueName: \"kubernetes.io/projected/e7561551-7ec6-43f6-9542-77c479bd3975-kube-api-access-q5hh9\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.891317 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.891287    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfj66\" (UniqueName: \"kubernetes.io/projected/2ebc48f1-c278-4176-9256-5e017ca78b2c-kube-api-access-xfj66\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:36.892565 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.892525    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fwjp\" (UniqueName: \"kubernetes.io/projected/ff5c6570-b247-497c-aa51-90e92c38db3c-kube-api-access-8fwjp\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.892740 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.892719    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqfx7\" (UniqueName: \"kubernetes.io/projected/fa1fb598-13ff-4b53-bec5-85cf10855d09-kube-api-access-fqfx7\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.892829 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.892809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrvx\" (UniqueName: \"kubernetes.io/projected/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-kube-api-access-phrvx\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.893141 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.893117    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brhp9\" (UniqueName: \"kubernetes.io/projected/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-kube-api-access-brhp9\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.893321 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.893303    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s522\" (UniqueName: \"kubernetes.io/projected/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-kube-api-access-7s522\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.893651 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.893633    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98skz\" (UniqueName: \"kubernetes.io/projected/f97dec7e-9808-4b45-8007-fbaacb1f65b5-kube-api-access-98skz\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.896518 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.896504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:22:36.901814 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.901794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:22:36.907448 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.907431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:36.907933 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.907905    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3055bb4_b2a7_4ae2_9f54_bc4f4a21a1c9.slice/crio-d200180791783c0aeea6a61373a0bc7dcd85b2e4e7ef54b429b4346d9c3c0616 WatchSource:0}: Error finding container d200180791783c0aeea6a61373a0bc7dcd85b2e4e7ef54b429b4346d9c3c0616: Status 404 returned error can't find the container with id d200180791783c0aeea6a61373a0bc7dcd85b2e4e7ef54b429b4346d9c3c0616
Nov 03 22:22:36.912263 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.912234    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51712151_3879_46f0_99fb_919861bf9ee5.slice/crio-8be0b40b8c59aa4f24a5ce1151e939106f645a679eaede40c617fe49ddbbf7b8 WatchSource:0}: Error finding container 8be0b40b8c59aa4f24a5ce1151e939106f645a679eaede40c617fe49ddbbf7b8: Status 404 returned error can't find the container with id 8be0b40b8c59aa4f24a5ce1151e939106f645a679eaede40c617fe49ddbbf7b8
Nov 03 22:22:36.915593 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.915578    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c344bf_cb2b_4eef_aaa5_c7ae74e8a982.slice/crio-82f1234aca9bb72f7064b9e6be00f4b8ed66c8d99535f3864d4160eae1eabf25 WatchSource:0}: Error finding container 82f1234aca9bb72f7064b9e6be00f4b8ed66c8d99535f3864d4160eae1eabf25: Status 404 returned error can't find the container with id 82f1234aca9bb72f7064b9e6be00f4b8ed66c8d99535f3864d4160eae1eabf25
Nov 03 22:22:36.916357 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.916341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:22:36.924430 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.924414    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1fb598_13ff_4b53_bec5_85cf10855d09.slice/crio-b998e15949dc74306d9555181b3fd47895f3651206de9bf11f0425acf40cd52a WatchSource:0}: Error finding container b998e15949dc74306d9555181b3fd47895f3651206de9bf11f0425acf40cd52a: Status 404 returned error can't find the container with id b998e15949dc74306d9555181b3fd47895f3651206de9bf11f0425acf40cd52a
Nov 03 22:22:36.929406 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.929379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6nlml"
Nov 03 22:22:36.936518 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.936500    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7561551_7ec6_43f6_9542_77c479bd3975.slice/crio-211fa1445ea540b35c0fd1a00a4bd1380fd6b777da0dcf430243d785b2577eb1 WatchSource:0}: Error finding container 211fa1445ea540b35c0fd1a00a4bd1380fd6b777da0dcf430243d785b2577eb1: Status 404 returned error can't find the container with id 211fa1445ea540b35c0fd1a00a4bd1380fd6b777da0dcf430243d785b2577eb1
Nov 03 22:22:36.937770 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.937752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:22:36.940204 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.940189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:22:36.944323 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.944302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:22:36.946247 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.946228    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7f40a7_30f2_48be_a46b_6221ef0b23d5.slice/crio-ee289e44e4381a36505fd421ce3b1cbe4cda58125e31c990f3acdc6387faf8da WatchSource:0}: Error finding container ee289e44e4381a36505fd421ce3b1cbe4cda58125e31c990f3acdc6387faf8da: Status 404 returned error can't find the container with id ee289e44e4381a36505fd421ce3b1cbe4cda58125e31c990f3acdc6387faf8da
Nov 03 22:22:36.949383 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.949365    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa38b7d_d2f8_439e_88f6_034a12d5b0d1.slice/crio-d17e7ac97721fb5b318182d6bc66b9fe2c0ab35705281414724690cb63d86cbc WatchSource:0}: Error finding container d17e7ac97721fb5b318182d6bc66b9fe2c0ab35705281414724690cb63d86cbc: Status 404 returned error can't find the container with id d17e7ac97721fb5b318182d6bc66b9fe2c0ab35705281414724690cb63d86cbc
Nov 03 22:22:36.953656 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:36.953638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:36.953849 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.953834    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5c6570_b247_497c_aa51_90e92c38db3c.slice/crio-45f5199f3ac226370e559e6dadba159fd251f9db973c0e7390aec0ae7fd46ab4 WatchSource:0}: Error finding container 45f5199f3ac226370e559e6dadba159fd251f9db973c0e7390aec0ae7fd46ab4: Status 404 returned error can't find the container with id 45f5199f3ac226370e559e6dadba159fd251f9db973c0e7390aec0ae7fd46ab4
Nov 03 22:22:36.961677 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:22:36.961657    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf97dec7e_9808_4b45_8007_fbaacb1f65b5.slice/crio-866f622b8a4b692af18665a9cfcb52bb4555c6473e779a9e952cd7df0ca1edb5 WatchSource:0}: Error finding container 866f622b8a4b692af18665a9cfcb52bb4555c6473e779a9e952cd7df0ca1edb5: Status 404 returned error can't find the container with id 866f622b8a4b692af18665a9cfcb52bb4555c6473e779a9e952cd7df0ca1edb5
Nov 03 22:22:37.335060 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.335035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:37.335530 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.335098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:37.335530 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.335193    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:37.335530 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.335205    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:37.335530 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.335253    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs podName:2ebc48f1-c278-4176-9256-5e017ca78b2c nodeName:}" failed. No retries permitted until 2026-11-03 22:22:38.335239808 +0000 UTC m=+23.414696687 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs") pod "network-metrics-daemon-glv5p" (UID: "2ebc48f1-c278-4176-9256-5e017ca78b2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:37.335530 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.335270    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret podName:48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:38.335261605 +0000 UTC m=+23.414718484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret") pod "global-pull-secret-syncer-86xxh" (UID: "48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:37.435755 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.435730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:37.435889 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.435837    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:37.435889 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.435852    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:37.435889 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.435861    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vp9qw for pod openshift-network-diagnostics/network-check-target-cvzp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:37.435993 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:37.435902    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw podName:571f0155-36fa-4eb3-9ddd-9317472e6761 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:38.435891403 +0000 UTC m=+23.515348281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vp9qw" (UniqueName: "kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw") pod "network-check-target-cvzp9" (UID: "571f0155-36fa-4eb3-9ddd-9317472e6761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:37.538927 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.538887    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"866f622b8a4b692af18665a9cfcb52bb4555c6473e779a9e952cd7df0ca1edb5"}
Nov 03 22:22:37.539916 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.539899    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kzxsv" event={"ID":"efa38b7d-d2f8-439e-88f6-034a12d5b0d1","Type":"ContainerStarted","Data":"d17e7ac97721fb5b318182d6bc66b9fe2c0ab35705281414724690cb63d86cbc"}
Nov 03 22:22:37.540897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.540878    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerStarted","Data":"ee289e44e4381a36505fd421ce3b1cbe4cda58125e31c990f3acdc6387faf8da"}
Nov 03 22:22:37.541865 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.541841    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6nlml" event={"ID":"e7561551-7ec6-43f6-9542-77c479bd3975","Type":"ContainerStarted","Data":"211fa1445ea540b35c0fd1a00a4bd1380fd6b777da0dcf430243d785b2577eb1"}
Nov 03 22:22:37.542740 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.542723    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j" event={"ID":"fa1fb598-13ff-4b53-bec5-85cf10855d09","Type":"ContainerStarted","Data":"b998e15949dc74306d9555181b3fd47895f3651206de9bf11f0425acf40cd52a"}
Nov 03 22:22:37.543630 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.543605    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dtsg" event={"ID":"ff5c6570-b247-497c-aa51-90e92c38db3c","Type":"ContainerStarted","Data":"45f5199f3ac226370e559e6dadba159fd251f9db973c0e7390aec0ae7fd46ab4"}
Nov 03 22:22:37.544522 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.544498    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d46vm" event={"ID":"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982","Type":"ContainerStarted","Data":"82f1234aca9bb72f7064b9e6be00f4b8ed66c8d99535f3864d4160eae1eabf25"}
Nov 03 22:22:37.545415 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.545383    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vr4x9" event={"ID":"51712151-3879-46f0-99fb-919861bf9ee5","Type":"ContainerStarted","Data":"8be0b40b8c59aa4f24a5ce1151e939106f645a679eaede40c617fe49ddbbf7b8"}
Nov 03 22:22:37.546470 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.546452    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5rr7s" event={"ID":"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9","Type":"ContainerStarted","Data":"d200180791783c0aeea6a61373a0bc7dcd85b2e4e7ef54b429b4346d9c3c0616"}
Nov 03 22:22:37.626444 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.626383    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 22:17:36 +0000 UTC" deadline="2028-08-20 05:23:33.310838988 +0000 UTC"
Nov 03 22:22:37.626513 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:37.626445    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15727h0m55.68439769s"
Nov 03 22:22:38.013137 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.011378    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal" podStartSLOduration=3.011359713 podStartE2EDuration="3.011359713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:38.010654457 +0000 UTC m=+23.090111358" watchObservedRunningTime="2026-11-03 22:22:38.011359713 +0000 UTC m=+23.090816617"
Nov 03 22:22:38.340884 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.340730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:38.340884 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.340802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:38.341430 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.340922    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:38.341430 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.340980    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret podName:48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:40.340962807 +0000 UTC m=+25.420419706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret") pod "global-pull-secret-syncer-86xxh" (UID: "48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:38.341430 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.341334    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:38.341430 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.341377    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs podName:2ebc48f1-c278-4176-9256-5e017ca78b2c nodeName:}" failed. No retries permitted until 2026-11-03 22:22:40.341364738 +0000 UTC m=+25.420821630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs") pod "network-metrics-daemon-glv5p" (UID: "2ebc48f1-c278-4176-9256-5e017ca78b2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:38.371716 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.371680    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:38.441280 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.441248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:38.441508 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.441419    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:38.441508 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.441442    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:38.441508 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.441455    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vp9qw for pod openshift-network-diagnostics/network-check-target-cvzp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:38.441508 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.441509    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw podName:571f0155-36fa-4eb3-9ddd-9317472e6761 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:40.441490461 +0000 UTC m=+25.520947346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vp9qw" (UniqueName: "kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw") pod "network-check-target-cvzp9" (UID: "571f0155-36fa-4eb3-9ddd-9317472e6761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:38.492801 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.492766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:38.492801 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.492806    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:38.493180 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.492905    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:38.493180 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.492956    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:38.493180 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.493023    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:38.493180 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.493102    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:38.572346 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.572301    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal_43c3a967cbec18dc888ae76406901416/setup/0.log"
Nov 03 22:22:38.572546 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.572349    2580 generic.go:356] "Generic (PLEG): container finished" podID="43c3a967cbec18dc888ae76406901416" containerID="97dbe1ff7ccbcc404d29d98640c63290b94ec8634808b2f5b49138b10aaeafff" exitCode=0
Nov 03 22:22:38.572546 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.572412    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerDied","Data":"97dbe1ff7ccbcc404d29d98640c63290b94ec8634808b2f5b49138b10aaeafff"}
Nov 03 22:22:38.572546 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.572482    2580 scope.go:122] "RemoveContainer" containerID="3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a"
Nov 03 22:22:38.572786 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.572777    2580 scope.go:122] "RemoveContainer" containerID="3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a"
Nov 03 22:22:38.624383 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:38.624340    2580 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal_openshift-machine-config-operator_43c3a967cbec18dc888ae76406901416_0 in pod sandbox fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957 from index: no such id: '3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a'" containerID="3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a"
Nov 03 22:22:38.624517 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.624412    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a"} err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal_openshift-machine-config-operator_43c3a967cbec18dc888ae76406901416_0 in pod sandbox fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957 from index: no such id: '3ece057507ffb7d9ed63aaf6b295ab7ba590a7bcd9bff2e78396216ea3413e5a'"
Nov 03 22:22:38.627025 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.626994    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 22:17:36 +0000 UTC" deadline="2028-07-12 23:18:55.878661256 +0000 UTC"
Nov 03 22:22:38.627110 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:38.627026    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14808h56m17.251638727s"
Nov 03 22:22:39.588936 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:39.588854    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerStarted","Data":"1fc55907adb0a44c70a0a83566482916ca917df7cda700db53aafb1232418414"}
Nov 03 22:22:40.360841 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.360802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:40.361000 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.360883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:40.361040 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.361012    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:40.361088 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.361076    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret podName:48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:44.361058545 +0000 UTC m=+29.440515445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret") pod "global-pull-secret-syncer-86xxh" (UID: "48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:40.361501 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.361481    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:40.361594 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.361539    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs podName:2ebc48f1-c278-4176-9256-5e017ca78b2c nodeName:}" failed. No retries permitted until 2026-11-03 22:22:44.361522587 +0000 UTC m=+29.440979469 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs") pod "network-metrics-daemon-glv5p" (UID: "2ebc48f1-c278-4176-9256-5e017ca78b2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:40.461628 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.461589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:40.461805 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.461753    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:40.461805 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.461772    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:40.461805 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.461783    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vp9qw for pod openshift-network-diagnostics/network-check-target-cvzp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:40.461976 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.461837    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw podName:571f0155-36fa-4eb3-9ddd-9317472e6761 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:44.461821029 +0000 UTC m=+29.541277907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vp9qw" (UniqueName: "kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw") pod "network-check-target-cvzp9" (UID: "571f0155-36fa-4eb3-9ddd-9317472e6761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:40.492901 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.492865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:40.493059 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.492986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:40.493132 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.493062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:40.493181 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.493131    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:40.493220 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.493183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:40.493280 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:40.493259    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:40.594562 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.594493    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="a7f2605858ef405299cdd7a9486c8c8fa72ac0fee2a55680421f62dfe27ecf53" exitCode=0
Nov 03 22:22:40.595027 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.594573    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerDied","Data":"a7f2605858ef405299cdd7a9486c8c8fa72ac0fee2a55680421f62dfe27ecf53"}
Nov 03 22:22:40.741782 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:40.740909    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" podStartSLOduration=5.7408891 podStartE2EDuration="5.7408891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:39.709366432 +0000 UTC m=+24.788823335" watchObservedRunningTime="2026-11-03 22:22:40.7408891 +0000 UTC m=+25.820346003"
Nov 03 22:22:41.606316 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:41.606069    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d46vm" event={"ID":"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982","Type":"ContainerStarted","Data":"8499e4ba738fcd4b12b0cf7a7175249d03bb699c39ec23f292d99d155ff92b9d"}
Nov 03 22:22:41.724654 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:41.724600    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d46vm" podStartSLOduration=8.661593344 podStartE2EDuration="12.724581002s" totalImagesPullingTime="4.062987658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:41.724191328 +0000 UTC m=+26.803648264" watchObservedRunningTime="2026-11-03 22:22:41.724581002 +0000 UTC m=+26.804037905"
Nov 03 22:22:41.908443 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:41.908039    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:41.909178 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:41.908928    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:42.492981 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:42.492947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:42.493151 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:42.493085    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:42.493512 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:42.493487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:42.493617 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:42.493582    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:42.493689 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:42.493635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:42.493741 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:42.493717    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:43.619751 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:43.616637    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j" event={"ID":"fa1fb598-13ff-4b53-bec5-85cf10855d09","Type":"ContainerStarted","Data":"3e071756f1b1107d2978cd066b03da129659e41fa936d1fd71cbe3ef2f81aa59"}
Nov 03 22:22:43.619751 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:43.618329    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 22:22:43.619751 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:43.619045    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dtsg" event={"ID":"ff5c6570-b247-497c-aa51-90e92c38db3c","Type":"ContainerStarted","Data":"687483a6d808750585ae4b60d6efb2d0b8bcb83673ffc2a615c7b6387eb2dace"}
Nov 03 22:22:44.399371 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:44.399331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:44.399548 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:44.399419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:44.399609 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.399563    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:44.399656 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.399621    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret podName:48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:52.399603562 +0000 UTC m=+37.479060464 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret") pod "global-pull-secret-syncer-86xxh" (UID: "48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:44.400083 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.399997    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:44.400083 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.400037    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs podName:2ebc48f1-c278-4176-9256-5e017ca78b2c nodeName:}" failed. No retries permitted until 2026-11-03 22:22:52.400026693 +0000 UTC m=+37.479483585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs") pod "network-metrics-daemon-glv5p" (UID: "2ebc48f1-c278-4176-9256-5e017ca78b2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:44.492831 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:44.492494    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:44.492831 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.492588    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:44.492831 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:44.492651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:44.492831 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.492696    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:44.492831 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:44.492736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:44.492831 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.492801    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:44.500734 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:44.500352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:44.500734 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.500526    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:44.500734 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.500550    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:44.500734 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.500562    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vp9qw for pod openshift-network-diagnostics/network-check-target-cvzp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:44.500734 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:44.500609    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw podName:571f0155-36fa-4eb3-9ddd-9317472e6761 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:52.500593384 +0000 UTC m=+37.580050272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vp9qw" (UniqueName: "kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw") pod "network-check-target-cvzp9" (UID: "571f0155-36fa-4eb3-9ddd-9317472e6761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:45.364268 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.364241    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 22:22:45.462234 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.462112    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T22:22:45.364265755Z","UUID":"5e7ff539-76dc-4fc4-88eb-1ee7ebfcb1e7","Handler":null,"Name":"","Endpoint":""}
Nov 03 22:22:45.463904 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.463881    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 22:22:45.463904 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.463909    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 22:22:45.623623 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.623533    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vr4x9" event={"ID":"51712151-3879-46f0-99fb-919861bf9ee5","Type":"ContainerStarted","Data":"2e30e99cfaa5e125ac26e465e11698167bf20260009887b24dd9876469097413"}
Nov 03 22:22:45.626886 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.626855    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6nlml" event={"ID":"e7561551-7ec6-43f6-9542-77c479bd3975","Type":"ContainerStarted","Data":"464b96d1cb19df7942e28c19d2878d1e3fadbf1e7a0bce4aeb08dcce031a2926"}
Nov 03 22:22:45.628869 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.628840    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j" event={"ID":"fa1fb598-13ff-4b53-bec5-85cf10855d09","Type":"ContainerStarted","Data":"8d61480ce3a694cae1a396ef597eae5bead2faaea43cf7549a219478156a47a2"}
Nov 03 22:22:45.741838 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.741642    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5dtsg" podStartSLOduration=11.036026215 podStartE2EDuration="16.741623789s" totalImagesPullingTime="5.705597574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:43.73701975 +0000 UTC m=+28.816476651" watchObservedRunningTime="2026-11-03 22:22:45.741623789 +0000 UTC m=+30.821080691"
Nov 03 22:22:45.863164 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.863111    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-vr4x9" podStartSLOduration=8.440400023 podStartE2EDuration="16.863097444s" totalImagesPullingTime="8.422697421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:45.742019187 +0000 UTC m=+30.821476081" watchObservedRunningTime="2026-11-03 22:22:45.863097444 +0000 UTC m=+30.942554344"
Nov 03 22:22:45.863647 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:45.863585    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-6nlml" podStartSLOduration=9.038475324 podStartE2EDuration="16.863572082s" totalImagesPullingTime="7.825096758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:45.863005889 +0000 UTC m=+30.942462791" watchObservedRunningTime="2026-11-03 22:22:45.863572082 +0000 UTC m=+30.943028984"
Nov 03 22:22:46.082783 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.082680    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:46.082953 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.082830    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 22:22:46.083488 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.083462    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:22:46.492301 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.492232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:46.492744 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:46.492367    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:46.492744 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.492712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:46.492860 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:46.492780    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:46.492860 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.492823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:46.492970 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:46.492863    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:46.634465 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.634424    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="0dd58c290f34545c64b4a501a35bcf65dca60ae4be0d9ff3338f8886cd73bff6" exitCode=0
Nov 03 22:22:46.635414 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:46.635375    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerDied","Data":"0dd58c290f34545c64b4a501a35bcf65dca60ae4be0d9ff3338f8886cd73bff6"}
Nov 03 22:22:47.639048 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:47.639010    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j" event={"ID":"fa1fb598-13ff-4b53-bec5-85cf10855d09","Type":"ContainerStarted","Data":"1c057abef023515f1eaaae066d5fa4a2681888e24fe2a904cac2a12c90bfd5c7"}
Nov 03 22:22:47.640695 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:47.640668    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5rr7s" event={"ID":"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9","Type":"ContainerStarted","Data":"292e881702d966b0cd422537f6f84300233c9080191353470cf1e79e49e8e25e"}
Nov 03 22:22:47.642584 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:47.642190    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kzxsv" event={"ID":"efa38b7d-d2f8-439e-88f6-034a12d5b0d1","Type":"ContainerStarted","Data":"762cf52aee624897761c798b53d325bec60edca337ab95e1f89ba352b47d895a"}
Nov 03 22:22:47.660183 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:47.660149    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:47.760178 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:47.760131    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j" podStartSLOduration=9.096577677 podStartE2EDuration="18.760119295s" totalImagesPullingTime="9.663541618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:47.759555133 +0000 UTC m=+32.839012046" watchObservedRunningTime="2026-11-03 22:22:47.760119295 +0000 UTC m=+32.839576254"
Nov 03 22:22:47.883883 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:47.883840    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5rr7s" podStartSLOduration=8.736311546 podStartE2EDuration="18.883823366s" totalImagesPullingTime="10.14751182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:47.883492926 +0000 UTC m=+32.962949828" watchObservedRunningTime="2026-11-03 22:22:47.883823366 +0000 UTC m=+32.963280268"
Nov 03 22:22:47.912385 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:47.912307    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:48.001401 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.001338    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kzxsv" podStartSLOduration=10.862385257 podStartE2EDuration="19.001318513s" totalImagesPullingTime="8.138933256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:48.000768787 +0000 UTC m=+33.080225708" watchObservedRunningTime="2026-11-03 22:22:48.001318513 +0000 UTC m=+33.080775415"
Nov 03 22:22:48.175843 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.175771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:48.375074 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.375039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:48.492861 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.492775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:48.492861 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.492836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:48.493036 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.492922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:48.493036 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:48.492934    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:48.493126 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:48.493027    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:48.493160 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:48.493117    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:48.572013 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.571987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:48.646374 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.646339    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="656bc224758e92f565820c7d72fe573bef3fbb395c8ce9c7dea71a6af6864541" exitCode=0
Nov 03 22:22:48.646803 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:48.646470    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerDied","Data":"656bc224758e92f565820c7d72fe573bef3fbb395c8ce9c7dea71a6af6864541"}
Nov 03 22:22:49.062234 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:49.062193    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:49.264938 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:49.264889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:49.536081 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:49.535997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:50.173429 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.173339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:50.493144 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.493060    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:50.493144 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.493091    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:50.493144 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.493102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:50.493421 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:50.493176    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:50.493421 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:50.493254    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:50.493421 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:50.493335    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:50.577887 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.577855    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:50.652429 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.652379    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="d7d4e4da786900d3f28d36b51ff87d1f5e5ec280b40ad07f7191a511b3657c86" exitCode=0
Nov 03 22:22:50.652429 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.652426    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerDied","Data":"d7d4e4da786900d3f28d36b51ff87d1f5e5ec280b40ad07f7191a511b3657c86"}
Nov 03 22:22:50.790666 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.790590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:50.988518 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:50.988486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:51.191001 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:51.190974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:51.657376 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:51.657208    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"e08590bc03c2b8c56be88caa25f6528d974c3e6bbcc7eee37cc4005efa1c054b"}
Nov 03 22:22:51.657530 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:51.657440    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"1be2da3786def2f57ab1e28728320dd7645c0862f6157a9979fbe826fb61d6a5"}
Nov 03 22:22:51.657530 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:51.657474    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"409e00586a55a59b25437ad1fb9cb7fbb86da4df3fe38c57248f69896262141f"}
Nov 03 22:22:51.657530 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:51.657505    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"f0b194d1d27f5750a10e188d5da18a2aa814ee4b03dd4b5c772c1cb7a28fac60"}
Nov 03 22:22:51.657619 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:51.657533    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"dfd92f61ddff42bfb18152c47106b6e98952022f7913a6599619fdbcbf504d48"}
Nov 03 22:22:51.657619 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:51.657559    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"b3cee03e067ec0d60f2cf73fff71f68329f2d4ed8c911dff4b72579689508bcf"}
Nov 03 22:22:52.456974 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.456937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:52.457684 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.457022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:52.457684 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.457080    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:52.457684 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.457143    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret podName:48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44 nodeName:}" failed. No retries permitted until 2026-11-03 22:23:08.457124645 +0000 UTC m=+53.536581530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret") pod "global-pull-secret-syncer-86xxh" (UID: "48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:52.457684 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.457154    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:52.457684 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.457208    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs podName:2ebc48f1-c278-4176-9256-5e017ca78b2c nodeName:}" failed. No retries permitted until 2026-11-03 22:23:08.457191595 +0000 UTC m=+53.536648475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs") pod "network-metrics-daemon-glv5p" (UID: "2ebc48f1-c278-4176-9256-5e017ca78b2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:52.493050 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.493023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:52.493184 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.493059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:52.493184 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.493145    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:52.493278 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.493183    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:52.493278 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.493244    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:52.493368 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.493353    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:52.557473 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.557446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:52.557631 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.557606    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:52.557631 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.557631    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:52.557815 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.557642    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vp9qw for pod openshift-network-diagnostics/network-check-target-cvzp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:52.557815 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:52.557696    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw podName:571f0155-36fa-4eb3-9ddd-9317472e6761 nodeName:}" failed. No retries permitted until 2026-11-03 22:23:08.55768308 +0000 UTC m=+53.637139960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vp9qw" (UniqueName: "kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw") pod "network-check-target-cvzp9" (UID: "571f0155-36fa-4eb3-9ddd-9317472e6761") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:52.599271 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.599237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:52.806138 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:52.806069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:53.036094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:53.036059    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:53.664911 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:53.664877    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"73aae62687656be64f87a41d70f21c70d3b0ea1a1aaff4a80267a41a4f25bf49"}
Nov 03 22:22:53.667492 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:53.667466    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="f261194e9962acc24541eb9fadd74b6f750518b0f1b2408a1033df259aa0a275" exitCode=0
Nov 03 22:22:53.667605 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:53.667508    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerDied","Data":"f261194e9962acc24541eb9fadd74b6f750518b0f1b2408a1033df259aa0a275"}
Nov 03 22:22:54.493150 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:54.493116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:54.493301 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:54.493175    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:54.493301 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:54.493201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:54.493301 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:54.493272    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:54.493438 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:54.493320    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:54.493438 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:54.493411    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:54.672465 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:54.672428    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="77c8f1eebd68341660f3081abfb8a2f7f462117657d4269796e43f0f728cbfba" exitCode=0
Nov 03 22:22:54.672811 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:54.672479    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerDied","Data":"77c8f1eebd68341660f3081abfb8a2f7f462117657d4269796e43f0f728cbfba"}
Nov 03 22:22:54.760468 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:54.760437    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:54.971954 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:54.971921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:55.678129 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:55.677978    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" event={"ID":"1a7f40a7-30f2-48be-a46b-6221ef0b23d5","Type":"ContainerStarted","Data":"7a317217310a099cf9d57a7dd8b23128c7857d56b1a4c8d9f6b323f6cabc4392"}
Nov 03 22:22:55.905613 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:55.905506    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qj4mh" podStartSLOduration=14.325213251 podStartE2EDuration="26.90549235s" totalImagesPullingTime="12.580279099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:55.85373622 +0000 UTC m=+40.933193121" watchObservedRunningTime="2026-11-03 22:22:55.90549235 +0000 UTC m=+40.984949251"
Nov 03 22:22:56.492580 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:56.492545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:56.492713 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:56.492630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:56.492763 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:56.492722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:56.492763 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:56.492727    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:56.492839 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:56.492818    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:56.492905 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:56.492890    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:56.683047 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:56.683013    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" event={"ID":"f97dec7e-9808-4b45-8007-fbaacb1f65b5","Type":"ContainerStarted","Data":"28e520928c8102c760a61c80b064d9f0c3b8cae254ec471512dfb94d7dfe6145"}
Nov 03 22:22:56.917502 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:56.917418    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf" podStartSLOduration=13.69641935 podStartE2EDuration="27.917404011s" totalImagesPullingTime="14.220984661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:56.865932311 +0000 UTC m=+41.945389212" watchObservedRunningTime="2026-11-03 22:22:56.917404011 +0000 UTC m=+41.996860906"
Nov 03 22:22:57.685785 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:57.685758    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:57.686293 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:57.685819    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:57.686293 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:57.685863    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:57.700824 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:57.700799    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:57.700979 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:57.700909    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:22:58.492654 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:58.492621    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:22:58.492654 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:58.492648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:22:58.492915 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:58.492745    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:22:58.492915 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:58.492807    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:22:58.492915 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:58.492844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:22:58.492915 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:22:58.492912    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:22:58.783100 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:58.783020    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:59.054901 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:59.054823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:59.253713 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:59.253677    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:59.516006 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:59.515974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:22:59.713717 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:22:59.713686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:00.492511 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.492483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:00.492925 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.492483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:00.492925 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.492604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:00.492925 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:00.492609    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:23:00.492925 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:00.492700    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:23:00.492925 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:00.492794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:23:00.513709 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.513685    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-86xxh"]
Nov 03 22:23:00.518018 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.517992    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cvzp9"]
Nov 03 22:23:00.518112 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.518050    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-glv5p"]
Nov 03 22:23:00.691798 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.691768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:00.691936 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.691768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:00.691936 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:00.691865    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:23:00.691936 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:00.691928    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:23:00.692051 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:00.691772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:00.692051 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:00.691998    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:23:02.492978 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:02.492953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:02.493347 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:02.492952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:02.493347 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:02.492954    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:02.493347 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:02.493124    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:23:02.493347 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:02.493040    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:23:02.493347 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:02.493195    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:23:03.303507 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:03.303303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:03.500184 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:03.500150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:03.703218 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:03.703192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:03.907232 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:03.907193    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:04.492806 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:04.492774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:04.492963 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:04.492853    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:04.492963 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:04.492933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:04.492963 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:04.492944    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-86xxh" podUID="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44"
Nov 03 22:23:04.493086 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:04.492993    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvzp9" podUID="571f0155-36fa-4eb3-9ddd-9317472e6761"
Nov 03 22:23:04.493086 ip-10-0-143-0 kubenswrapper[2580]: E1103 22:23:04.493069    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-glv5p" podUID="2ebc48f1-c278-4176-9256-5e017ca78b2c"
Nov 03 22:23:05.572988 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.572950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:05.734371 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.734346    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeReady"
Nov 03 22:23:05.734531 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.734463    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 22:23:05.768121 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.768095    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:05.852248 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.852217    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8lb4l"]
Nov 03 22:23:05.869205 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.869177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:05.903404 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.903367    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d5h8h"]
Nov 03 22:23:05.914005 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.913985    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:05.924532 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.924515    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v9lfz\""
Nov 03 22:23:05.924627 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.924581    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 22:23:05.924627 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.924580    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 22:23:05.950919 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.950896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248f403e-7506-48d0-9fa2-ed5c621df5fe-tmp-dir\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:05.950919 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.950921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248f403e-7506-48d0-9fa2-ed5c621df5fe-metrics-tls\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:05.951050 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.950939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8dae706-de9e-4dc8-868f-961ee5a24428-data-volume\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:05.951050 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.950962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8dae706-de9e-4dc8-868f-961ee5a24428-crio-socket\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:05.951050 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.951046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:05.951160 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.951095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bz7p\" (UniqueName: \"kubernetes.io/projected/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-api-access-5bz7p\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:05.951160 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.951142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248f403e-7506-48d0-9fa2-ed5c621df5fe-config-volume\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:05.951226 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.951170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lspjx\" (UniqueName: \"kubernetes.io/projected/248f403e-7506-48d0-9fa2-ed5c621df5fe-kube-api-access-lspjx\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:05.951226 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.951186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8dae706-de9e-4dc8-868f-961ee5a24428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:05.969819 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.969791    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8lb4l"]
Nov 03 22:23:05.970404 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.970373    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d5h8h"]
Nov 03 22:23:05.979272 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.979250    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mr7r8"]
Nov 03 22:23:05.996870 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:05.996822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.024705 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.024655    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 22:23:06.024705 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.024664    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 22:23:06.024705 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.024680    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 22:23:06.024705 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.024684    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 22:23:06.024867 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.024718    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vkjzf\""
Nov 03 22:23:06.045118 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.045096    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mr7r8"]
Nov 03 22:23:06.051535 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8dae706-de9e-4dc8-868f-961ee5a24428-data-volume\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.051625 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8dae706-de9e-4dc8-868f-961ee5a24428-crio-socket\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.051625 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.051625 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bz7p\" (UniqueName: \"kubernetes.io/projected/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-api-access-5bz7p\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.051736 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248f403e-7506-48d0-9fa2-ed5c621df5fe-config-volume\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.051736 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-cert\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.051736 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lspjx\" (UniqueName: \"kubernetes.io/projected/248f403e-7506-48d0-9fa2-ed5c621df5fe-kube-api-access-lspjx\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.051736 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8dae706-de9e-4dc8-868f-961ee5a24428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.051929 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051762    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8dae706-de9e-4dc8-868f-961ee5a24428-crio-socket\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.051929 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248f403e-7506-48d0-9fa2-ed5c621df5fe-tmp-dir\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.051929 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248f403e-7506-48d0-9fa2-ed5c621df5fe-metrics-tls\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.051929 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hs9m\" (UniqueName: \"kubernetes.io/projected/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-kube-api-access-7hs9m\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.051929 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.051897    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8dae706-de9e-4dc8-868f-961ee5a24428-data-volume\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.052160 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.052085    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248f403e-7506-48d0-9fa2-ed5c621df5fe-tmp-dir\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.052293 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.052269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248f403e-7506-48d0-9fa2-ed5c621df5fe-config-volume\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.052345 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.052269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.055684 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.055665    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8dae706-de9e-4dc8-868f-961ee5a24428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.056695 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.056676    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248f403e-7506-48d0-9fa2-ed5c621df5fe-metrics-tls\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.118320 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.118289    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bz7p\" (UniqueName: \"kubernetes.io/projected/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-api-access-5bz7p\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.118320 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.118314    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lspjx\" (UniqueName: \"kubernetes.io/projected/248f403e-7506-48d0-9fa2-ed5c621df5fe-kube-api-access-lspjx\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.124872 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.124851    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 22:23:06.124975 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.124946    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 22:23:06.125076 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.125063    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-2jqrs\""
Nov 03 22:23:06.125076 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.125067    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 22:23:06.152380 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.152357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-cert\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.152465 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.152449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hs9m\" (UniqueName: \"kubernetes.io/projected/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-kube-api-access-7hs9m\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.154468 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.154449    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-cert\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.196844 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.196815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:06.214297 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.214272    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hs9m\" (UniqueName: \"kubernetes.io/projected/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-kube-api-access-7hs9m\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.252246 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.252168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:06.316750 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.316531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:06.492683 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.492643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:06.492850 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.492647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:06.492850 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.492785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:06.547502 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.547477    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 22:23:06.547502 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.547477    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kbfvs\""
Nov 03 22:23:06.547631 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.547484    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 22:23:06.547631 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.547576    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 22:23:06.547631 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.547585    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lk9bw\""
Nov 03 22:23:06.547631 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.547589    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 22:23:06.547753 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.547637    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2dn77\""
Nov 03 22:23:06.635171 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:06.635145    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248f403e_7506_48d0_9fa2_ed5c621df5fe.slice/crio-d57b851a7522e2ec84fc26d236da95894ebf20a5c600959c376029aa87d2f2b5 WatchSource:0}: Error finding container d57b851a7522e2ec84fc26d236da95894ebf20a5c600959c376029aa87d2f2b5: Status 404 returned error can't find the container with id d57b851a7522e2ec84fc26d236da95894ebf20a5c600959c376029aa87d2f2b5
Nov 03 22:23:06.637062 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.637032    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8lb4l"]
Nov 03 22:23:06.665896 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:06.665873    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8dae706_de9e_4dc8_868f_961ee5a24428.slice/crio-80f3faaef361a84fd4a85c3449faddada50047c3fbd1a31881369b6ec0f02a9a WatchSource:0}: Error finding container 80f3faaef361a84fd4a85c3449faddada50047c3fbd1a31881369b6ec0f02a9a: Status 404 returned error can't find the container with id 80f3faaef361a84fd4a85c3449faddada50047c3fbd1a31881369b6ec0f02a9a
Nov 03 22:23:06.666623 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.666600    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d5h8h"]
Nov 03 22:23:06.705617 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.705591    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8lb4l" event={"ID":"248f403e-7506-48d0-9fa2-ed5c621df5fe","Type":"ContainerStarted","Data":"d57b851a7522e2ec84fc26d236da95894ebf20a5c600959c376029aa87d2f2b5"}
Nov 03 22:23:06.706637 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.706606    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d5h8h" event={"ID":"f8dae706-de9e-4dc8-868f-961ee5a24428","Type":"ContainerStarted","Data":"80f3faaef361a84fd4a85c3449faddada50047c3fbd1a31881369b6ec0f02a9a"}
Nov 03 22:23:06.712480 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:06.712459    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba7c426_a1b7_4c37_ba18_8e3fd21dab83.slice/crio-b8397cecdf1842716df3cb46db086a53b12abe7166033409494aba2826002d27 WatchSource:0}: Error finding container b8397cecdf1842716df3cb46db086a53b12abe7166033409494aba2826002d27: Status 404 returned error can't find the container with id b8397cecdf1842716df3cb46db086a53b12abe7166033409494aba2826002d27
Nov 03 22:23:06.714703 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.714685    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mr7r8"]
Nov 03 22:23:06.912522 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:06.912489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:07.113459 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.113373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:07.591103 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.591074    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-b495744d6-2wdfb"]
Nov 03 22:23:07.612087 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.612068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.654546 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.654519    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b495744d6-2wdfb"]
Nov 03 22:23:07.663111 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.663088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.663222 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.663115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.663222 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.663136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.663222 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.663154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.663316 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.663301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.663350 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.663329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.663380 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.663351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65p2p\" (UniqueName: \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.666407 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666360    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 22:23:07.666519 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666411    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 22:23:07.666519 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666456    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 22:23:07.666519 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666464    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 22:23:07.666519 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666476    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 22:23:07.666715 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666627    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 22:23:07.666715 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666644    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-89fmh\""
Nov 03 22:23:07.666715 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.666662    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 22:23:07.710798 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.710768    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d5h8h" event={"ID":"f8dae706-de9e-4dc8-868f-961ee5a24428","Type":"ContainerStarted","Data":"565e9b78ee157d6494cdce86330fcfbd3c9120c17b77bc3202c4a6c004c52bea"}
Nov 03 22:23:07.711808 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.711788    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mr7r8" event={"ID":"fba7c426-a1b7-4c37-ba18-8e3fd21dab83","Type":"ContainerStarted","Data":"b8397cecdf1842716df3cb46db086a53b12abe7166033409494aba2826002d27"}
Nov 03 22:23:07.764689 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.764664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65p2p\" (UniqueName: \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.764805 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.764719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.764805 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.764742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.764805 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.764786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.764959 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.764821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.764959 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.764887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.764959 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.764916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.765312 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.765291    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.765634 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.765610    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.765721 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.765634    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.765721 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.765647    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.766994 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.766975    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.767056 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.767039    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.827449 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.827424    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65p2p\" (UniqueName: \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:07.935372 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:07.935327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:08.325082 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.325048    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b495744d6-2wdfb"]
Nov 03 22:23:08.326253 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:08.326226    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940f07aa_e2a5_4063_a837_2237c8585bb8.slice/crio-954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156 WatchSource:0}: Error finding container 954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156: Status 404 returned error can't find the container with id 954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156
Nov 03 22:23:08.470665 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.470627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:08.470844 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.470709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:08.473423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.473385    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:08.473550 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.473514    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:08.572101 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.572010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:08.575276 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.575239    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:08.616498 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.616466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:08.624564 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.624519    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:08.643298 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.643270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:08.715790 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:08.715755    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b495744d6-2wdfb" event={"ID":"940f07aa-e2a5-4063-a837-2237c8585bb8","Type":"ContainerStarted","Data":"954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156"}
Nov 03 22:23:09.190163 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.190124    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-glv5p"]
Nov 03 22:23:09.229148 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:09.224628    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ebc48f1_c278_4176_9256_5e017ca78b2c.slice/crio-3913665de6bc0489d510addc3fb2d23170c23b9c1866e93ef5a410629f17adf8 WatchSource:0}: Error finding container 3913665de6bc0489d510addc3fb2d23170c23b9c1866e93ef5a410629f17adf8: Status 404 returned error can't find the container with id 3913665de6bc0489d510addc3fb2d23170c23b9c1866e93ef5a410629f17adf8
Nov 03 22:23:09.239482 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:09.239453    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod571f0155_36fa_4eb3_9ddd_9317472e6761.slice/crio-375887f65d35a2142cea7f56c5ceb0aa08b449f9d89b5c9fbd90ec905e3cde14 WatchSource:0}: Error finding container 375887f65d35a2142cea7f56c5ceb0aa08b449f9d89b5c9fbd90ec905e3cde14: Status 404 returned error can't find the container with id 375887f65d35a2142cea7f56c5ceb0aa08b449f9d89b5c9fbd90ec905e3cde14
Nov 03 22:23:09.239730 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.239711    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-86xxh"]
Nov 03 22:23:09.240380 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.240359    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cvzp9"]
Nov 03 22:23:09.242076 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:09.242056    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ed2fb2_b25c_48fd_84c3_59a9a1ec4d44.slice/crio-4f64aecbb47d76b6ca7409f83c2733481b211c436841337fdf040a28b94b9a47 WatchSource:0}: Error finding container 4f64aecbb47d76b6ca7409f83c2733481b211c436841337fdf040a28b94b9a47: Status 404 returned error can't find the container with id 4f64aecbb47d76b6ca7409f83c2733481b211c436841337fdf040a28b94b9a47
Nov 03 22:23:09.595601 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.595518    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:09.720118 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.720093    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cvzp9" event={"ID":"571f0155-36fa-4eb3-9ddd-9317472e6761","Type":"ContainerStarted","Data":"375887f65d35a2142cea7f56c5ceb0aa08b449f9d89b5c9fbd90ec905e3cde14"}
Nov 03 22:23:09.721453 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.721424    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-glv5p" event={"ID":"2ebc48f1-c278-4176-9256-5e017ca78b2c","Type":"ContainerStarted","Data":"3913665de6bc0489d510addc3fb2d23170c23b9c1866e93ef5a410629f17adf8"}
Nov 03 22:23:09.723897 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.723869    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d5h8h" event={"ID":"f8dae706-de9e-4dc8-868f-961ee5a24428","Type":"ContainerStarted","Data":"2345a2e773d6346709a3ae6d8b6741b2d5b40e49d7779688cfcd82b07c845e83"}
Nov 03 22:23:09.725074 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.725052    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-86xxh" event={"ID":"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44","Type":"ContainerStarted","Data":"4f64aecbb47d76b6ca7409f83c2733481b211c436841337fdf040a28b94b9a47"}
Nov 03 22:23:09.800573 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:09.800544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:10.010907 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.010843    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:10.165199 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.165165    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"]
Nov 03 22:23:10.167666 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.167645    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.212215 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.212179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:10.222476 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.222099    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 22:23:10.222660 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.222567    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 22:23:10.222949 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.222871    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5v4d5\""
Nov 03 22:23:10.222949 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.222887    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 03 22:23:10.222949 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.222939    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 22:23:10.273342 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.273313    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 03 22:23:10.274383 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.274357    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"]
Nov 03 22:23:10.284265 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.284223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcfn9\" (UniqueName: \"kubernetes.io/projected/8086e88f-4770-416e-93fd-cf324fae5917-kube-api-access-xcfn9\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.284420 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.284287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8086e88f-4770-416e-93fd-cf324fae5917-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.284420 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.284323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.284420 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.284362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.304555 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.304529    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jfrn4"]
Nov 03 22:23:10.311495 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.311471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.369606 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.369578    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 22:23:10.369781 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.369725    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c2kdj\""
Nov 03 22:23:10.369871 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.369793    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 22:23:10.385245 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.385213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8086e88f-4770-416e-93fd-cf324fae5917-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.385371 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.385263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-root\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.385371 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.385293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfn9\" (UniqueName: \"kubernetes.io/projected/8086e88f-4770-416e-93fd-cf324fae5917-kube-api-access-xcfn9\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.385371 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.385324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.385371 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.385356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.386125 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386104    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8086e88f-4770-416e-93fd-cf324fae5917-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.386225 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-sys\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.386225 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.386320 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.386320 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-metrics-client-ca\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.386436 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-textfile\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.386436 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-tls\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.386538 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pjm9\" (UniqueName: \"kubernetes.io/projected/ec301699-cf86-453b-872b-b874de2dcf0d-kube-api-access-6pjm9\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.386538 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.386479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-wtmp\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.390002 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.389949    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.391453 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.391430    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.412620 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.412554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:10.424994 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.424962    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 22:23:10.455560 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.455527    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcfn9\" (UniqueName: \"kubernetes.io/projected/8086e88f-4770-416e-93fd-cf324fae5917-kube-api-access-xcfn9\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.484308 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.484282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:10.486859 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.486837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.486947 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.486885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-sys\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.486947 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.486915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.486947 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.486941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-metrics-client-ca\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.486970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-textfile\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-tls\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pjm9\" (UniqueName: \"kubernetes.io/projected/ec301699-cf86-453b-872b-b874de2dcf0d-kube-api-access-6pjm9\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-wtmp\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487095 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-root\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487318 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487176    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-root\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487714 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487696    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-textfile\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487807 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487759    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-sys\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.487868 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.487836    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.488546 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.488525    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-metrics-client-ca\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.488950 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.488927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-wtmp\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.490847 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.490827    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.492790 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.492736    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-tls\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.565270 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.565227    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pjm9\" (UniqueName: \"kubernetes.io/projected/ec301699-cf86-453b-872b-b874de2dcf0d-kube-api-access-6pjm9\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.616006 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.615923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:10.627990 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.627759    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:10.663045 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:10.663009    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec301699_cf86_453b_872b_b874de2dcf0d.slice/crio-835726585c2fb5bdd4dbdbcb5c135f9dd558a4ec18a7a1375b03047a269b94a6 WatchSource:0}: Error finding container 835726585c2fb5bdd4dbdbcb5c135f9dd558a4ec18a7a1375b03047a269b94a6: Status 404 returned error can't find the container with id 835726585c2fb5bdd4dbdbcb5c135f9dd558a4ec18a7a1375b03047a269b94a6
Nov 03 22:23:10.731780 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.730908    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8lb4l" event={"ID":"248f403e-7506-48d0-9fa2-ed5c621df5fe","Type":"ContainerStarted","Data":"edda138fd64704d345841f35ff27775810ac88450b449fd2cc413b78a5377482"}
Nov 03 22:23:10.731780 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.730972    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8lb4l" event={"ID":"248f403e-7506-48d0-9fa2-ed5c621df5fe","Type":"ContainerStarted","Data":"78f9f1ea618aabff7c659a93b97296f7f62f37bb87e0af5bcf70ced45836a366"}
Nov 03 22:23:10.731780 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.731715    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:10.733791 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.733711    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfrn4" event={"ID":"ec301699-cf86-453b-872b-b874de2dcf0d","Type":"ContainerStarted","Data":"835726585c2fb5bdd4dbdbcb5c135f9dd558a4ec18a7a1375b03047a269b94a6"}
Nov 03 22:23:10.829089 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:10.829013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:11.006926 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.006871    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-8lb4l" podStartSLOduration=2.925262414 podStartE2EDuration="6.006848532s" totalImagesPullingTime="3.081586118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:10.853047073 +0000 UTC m=+55.932503973" watchObservedRunningTime="2026-11-03 22:23:11.006848532 +0000 UTC m=+56.086305435"
Nov 03 22:23:11.007369 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.007345    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"]
Nov 03 22:23:11.009191 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:11.009162    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8086e88f_4770_416e_93fd_cf324fae5917.slice/crio-fa9b3c968c9181a97f069d4012e0efdb5493aba639f41f821c82f1be600f1f7c WatchSource:0}: Error finding container fa9b3c968c9181a97f069d4012e0efdb5493aba639f41f821c82f1be600f1f7c: Status 404 returned error can't find the container with id fa9b3c968c9181a97f069d4012e0efdb5493aba639f41f821c82f1be600f1f7c
Nov 03 22:23:11.024879 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.024849    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:11.354827 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.354749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:11.712689 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.712572    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:11.745222 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.745150    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-glv5p" event={"ID":"2ebc48f1-c278-4176-9256-5e017ca78b2c","Type":"ContainerStarted","Data":"f51e70ad75dbe5a12831e8406e1e72caf386a75a1db25c119fc22dec92ad2200"}
Nov 03 22:23:11.745672 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.745251    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-glv5p" event={"ID":"2ebc48f1-c278-4176-9256-5e017ca78b2c","Type":"ContainerStarted","Data":"f804be4d76252558a88dc87f7588d3fcf591bef44890c49ef2a4689d5e1f2008"}
Nov 03 22:23:11.748409 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.748376    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf" event={"ID":"8086e88f-4770-416e-93fd-cf324fae5917","Type":"ContainerStarted","Data":"9aba657dceb3b69cd54793308f9be84b3f74fd6ab3eb952cce432185effc8c6e"}
Nov 03 22:23:11.748528 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.748432    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf" event={"ID":"8086e88f-4770-416e-93fd-cf324fae5917","Type":"ContainerStarted","Data":"7d8f2e18bafeed076a21e3127fefd2a0b592b5a552c27a6d07761d8a7f223938"}
Nov 03 22:23:11.748528 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.748451    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf" event={"ID":"8086e88f-4770-416e-93fd-cf324fae5917","Type":"ContainerStarted","Data":"fa9b3c968c9181a97f069d4012e0efdb5493aba639f41f821c82f1be600f1f7c"}
Nov 03 22:23:11.752924 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.752810    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mr7r8" event={"ID":"fba7c426-a1b7-4c37-ba18-8e3fd21dab83","Type":"ContainerStarted","Data":"2bece9cdab1fc09f2252e862e62060354165513b4424a42da4f392b5be60759a"}
Nov 03 22:23:11.866463 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.865960    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-glv5p" podStartSLOduration=40.650242212 podStartE2EDuration="42.865910051s" totalImagesPullingTime="2.215667839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:11.864720469 +0000 UTC m=+56.944177382" watchObservedRunningTime="2026-11-03 22:23:11.865910051 +0000 UTC m=+56.945366955"
Nov 03 22:23:11.985581 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.985511    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mr7r8" podStartSLOduration=2.755113829 podStartE2EDuration="6.985468821s" totalImagesPullingTime="4.230354992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:11.984936622 +0000 UTC m=+57.064393525" watchObservedRunningTime="2026-11-03 22:23:11.985468821 +0000 UTC m=+57.064925726"
Nov 03 22:23:11.987617 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:11.987562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:12.186617 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.186537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:12.581423 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.581383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:12.759437 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.759381    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d5h8h" event={"ID":"f8dae706-de9e-4dc8-868f-961ee5a24428","Type":"ContainerStarted","Data":"82c1193972ef065a96f06a6059bf8c625cca807d407ebdcfa9c5d7df02130def"}
Nov 03 22:23:12.764285 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.764183    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfrn4" event={"ID":"ec301699-cf86-453b-872b-b874de2dcf0d","Type":"ContainerStarted","Data":"66af4e1d914d581713c078d9e18eb226ce1e9c26b645c2e1cde74e9debfb0f6a"}
Nov 03 22:23:12.883082 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.882355    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d5h8h" podStartSLOduration=2.832713541 podStartE2EDuration="7.882341929s" totalImagesPullingTime="5.049628388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:23:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:12.88105513 +0000 UTC m=+57.960512030" watchObservedRunningTime="2026-11-03 22:23:12.882341929 +0000 UTC m=+57.961798830"
Nov 03 22:23:12.902585 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.902128    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"]
Nov 03 22:23:12.906022 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.906004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:12.960681 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.960652    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 03 22:23:12.960843 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.960709    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 03 22:23:12.960843 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.960807    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 03 22:23:12.961021 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.961000    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 03 22:23:12.961116 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.961050    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 03 22:23:12.961232 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.961217    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d9kpm\""
Nov 03 22:23:12.961310 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.961273    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fodrd8fhtnt8d\""
Nov 03 22:23:12.968357 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:12.968333    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"]
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.008998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8kdc\" (UniqueName: \"kubernetes.io/projected/1695e3d3-d78a-41f2-bbad-67127b64a8fe-kube-api-access-r8kdc\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1695e3d3-d78a-41f2-bbad-67127b64a8fe-tmp\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1695e3d3-d78a-41f2-bbad-67127b64a8fe-metrics-client-ca\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-grpc-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.009422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.009223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8kdc\" (UniqueName: \"kubernetes.io/projected/1695e3d3-d78a-41f2-bbad-67127b64a8fe-kube-api-access-r8kdc\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1695e3d3-d78a-41f2-bbad-67127b64a8fe-tmp\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1695e3d3-d78a-41f2-bbad-67127b64a8fe-metrics-client-ca\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.110445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-grpc-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111631 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.111199    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1695e3d3-d78a-41f2-bbad-67127b64a8fe-tmp\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.111631 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.111524    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1695e3d3-d78a-41f2-bbad-67127b64a8fe-metrics-client-ca\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.114771 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.114685    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.114997 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.114971    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.115312 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.115287    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.115478 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.115414    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-grpc-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.115887 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.115855    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.116462 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.116437    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.174745 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.174718    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8kdc\" (UniqueName: \"kubernetes.io/projected/1695e3d3-d78a-41f2-bbad-67127b64a8fe-kube-api-access-r8kdc\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.229729 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.229694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:13.665611 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.665583    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"]
Nov 03 22:23:13.774887 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.774845    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf" event={"ID":"8086e88f-4770-416e-93fd-cf324fae5917","Type":"ContainerStarted","Data":"5c6980b545dcf2d07ae9c3462ae68e6c391f0c580c54ce67bf4147a7fd917c79"}
Nov 03 22:23:13.776646 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.776623    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2" event={"ID":"1695e3d3-d78a-41f2-bbad-67127b64a8fe","Type":"ContainerStarted","Data":"666e0b517c01cb2264be5dac231762cfb2aeaedd3e9a956ec726166cfa1b424d"}
Nov 03 22:23:13.778905 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.778879    2580 generic.go:356] "Generic (PLEG): container finished" podID="ec301699-cf86-453b-872b-b874de2dcf0d" containerID="66af4e1d914d581713c078d9e18eb226ce1e9c26b645c2e1cde74e9debfb0f6a" exitCode=0
Nov 03 22:23:13.779130 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.778926    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfrn4" event={"ID":"ec301699-cf86-453b-872b-b874de2dcf0d","Type":"ContainerDied","Data":"66af4e1d914d581713c078d9e18eb226ce1e9c26b645c2e1cde74e9debfb0f6a"}
Nov 03 22:23:13.779299 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.779281    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfrn4" event={"ID":"ec301699-cf86-453b-872b-b874de2dcf0d","Type":"ContainerStarted","Data":"36cd8b2798bab48d36502db98a93f33bc57b97a44f66f0af938aa7e17a3e7dd3"}
Nov 03 22:23:13.779459 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.779440    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jfrn4" event={"ID":"ec301699-cf86-453b-872b-b874de2dcf0d","Type":"ContainerStarted","Data":"e0f0e7a0421a2c1520e8df4d5bf72ce7d6d109045048cfecede04945f50247b2"}
Nov 03 22:23:13.782372 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.781897    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b495744d6-2wdfb" event={"ID":"940f07aa-e2a5-4063-a837-2237c8585bb8","Type":"ContainerStarted","Data":"f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2"}
Nov 03 22:23:13.897538 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:13.897433    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf" podStartSLOduration=1.983060136 podStartE2EDuration="3.89741276s" totalImagesPullingTime="1.914352624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:13.896353334 +0000 UTC m=+58.975810234" watchObservedRunningTime="2026-11-03 22:23:13.89741276 +0000 UTC m=+58.976869664"
Nov 03 22:23:14.019313 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.019257    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jfrn4" podStartSLOduration=2.033811308 podStartE2EDuration="4.019238521s" totalImagesPullingTime="1.903461337s" totalInitContainerRuntime="81.965876ms" isStatefulPod=true podCreationTimestamp="2026-11-03 22:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:14.018156547 +0000 UTC m=+59.097613449" watchObservedRunningTime="2026-11-03 22:23:14.019238521 +0000 UTC m=+59.098695425"
Nov 03 22:23:14.139746 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.139692    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-b495744d6-2wdfb" podStartSLOduration=1.8694197030000002 podStartE2EDuration="7.139676474s" totalImagesPullingTime="5.270256771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:14.138631748 +0000 UTC m=+59.218088649" watchObservedRunningTime="2026-11-03 22:23:14.139676474 +0000 UTC m=+59.219133374"
Nov 03 22:23:14.468719 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.468685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:14.731310 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.731271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:14.787649 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.787612    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cvzp9" event={"ID":"571f0155-36fa-4eb3-9ddd-9317472e6761","Type":"ContainerStarted","Data":"e73b885196cb27eb6b4de1652095d997391842c9ebb32d17e77ddb03f4d75a06"}
Nov 03 22:23:14.788194 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.787780    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:14.906640 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.906586    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cvzp9" podStartSLOduration=40.861636451 podStartE2EDuration="45.90657105s" totalImagesPullingTime="5.044934599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:14.905812004 +0000 UTC m=+59.985268906" watchObservedRunningTime="2026-11-03 22:23:14.90657105 +0000 UTC m=+59.986027951"
Nov 03 22:23:14.932465 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:14.932434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:15.010441 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.010355    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"]
Nov 03 22:23:15.013534 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.013108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.069682 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.069651    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 03 22:23:15.071718 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.071694    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"]
Nov 03 22:23:15.071834 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.071801    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-57qd7\""
Nov 03 22:23:15.131798 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.131765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:15.133207 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.133181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcjsw\" (UniqueName: \"kubernetes.io/projected/a9ae4421-28f8-420f-ade1-c91aafe026ab-kube-api-access-gcjsw\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.133341 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.133218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9ae4421-28f8-420f-ade1-c91aafe026ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.214211 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.214181    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"]
Nov 03 22:23:15.216734 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.216708    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.234378 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.234342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9ae4421-28f8-420f-ade1-c91aafe026ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.234493 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.234437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjsw\" (UniqueName: \"kubernetes.io/projected/a9ae4421-28f8-420f-ade1-c91aafe026ab-kube-api-access-gcjsw\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.237331 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.237309    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9ae4421-28f8-420f-ade1-c91aafe026ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.271325 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.271269    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 03 22:23:15.271460 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.271359    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 03 22:23:15.271460 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.271424    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 03 22:23:15.271529 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.271518    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 03 22:23:15.271668 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.271656    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vw4kb\""
Nov 03 22:23:15.271733 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.271723    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6iiok1m9m0lt2\""
Nov 03 22:23:15.277099 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.277078    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"]
Nov 03 22:23:15.297502 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.297456    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjsw\" (UniqueName: \"kubernetes.io/projected/a9ae4421-28f8-420f-ade1-c91aafe026ab-kube-api-access-gcjsw\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.334308 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.334244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:15.334844 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.334825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wldl\" (UniqueName: \"kubernetes.io/projected/16d2011b-252e-48d6-be1e-a1a9a8537b39-kube-api-access-6wldl\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.334935 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.334856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-metrics-server-audit-profiles\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.334935 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.334875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d2011b-252e-48d6-be1e-a1a9a8537b39-audit-log\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.335047 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.334945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-client-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.335102 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.335045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-client-certs\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.335102 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.335086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-tls\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.335198 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.335110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.380001 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.379969    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-57qd7\""
Nov 03 22:23:15.386511 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.386487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:15.436951 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.436912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-metrics-server-audit-profiles\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.437080 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.436975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d2011b-252e-48d6-be1e-a1a9a8537b39-audit-log\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.437080 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.437060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-client-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.437152 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.437117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-client-certs\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.437207 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.437192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-tls\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.437239 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.437224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.437299 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.437285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wldl\" (UniqueName: \"kubernetes.io/projected/16d2011b-252e-48d6-be1e-a1a9a8537b39-kube-api-access-6wldl\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.437899 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.437877    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d2011b-252e-48d6-be1e-a1a9a8537b39-audit-log\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.493012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.490261    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 03 22:23:15.493012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.492509    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 03 22:23:15.493012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.492437    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 03 22:23:15.493012 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.492859    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6iiok1m9m0lt2\""
Nov 03 22:23:15.493251 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.493171    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 03 22:23:15.502422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.499017    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.502422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.499799    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-metrics-server-audit-profiles\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.502422 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.501581    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wldl\" (UniqueName: \"kubernetes.io/projected/16d2011b-252e-48d6-be1e-a1a9a8537b39-kube-api-access-6wldl\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.504590 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.504564    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-client-certs\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.507867 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.507847    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-client-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.508019 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.507995    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-tls\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.582456 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.582352    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vw4kb\""
Nov 03 22:23:15.589878 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.589641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:15.778563 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:15.778528    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ae4421_28f8_420f_ade1_c91aafe026ab.slice/crio-eeb321ea37f35ba1eb4d9f05f6e38c31ff03e2e1fc75c13abdb796065ab0404c WatchSource:0}: Error finding container eeb321ea37f35ba1eb4d9f05f6e38c31ff03e2e1fc75c13abdb796065ab0404c: Status 404 returned error can't find the container with id eeb321ea37f35ba1eb4d9f05f6e38c31ff03e2e1fc75c13abdb796065ab0404c
Nov 03 22:23:15.780235 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.780209    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"]
Nov 03 22:23:15.793118 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.792989    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc" event={"ID":"a9ae4421-28f8-420f-ade1-c91aafe026ab","Type":"ContainerStarted","Data":"eeb321ea37f35ba1eb4d9f05f6e38c31ff03e2e1fc75c13abdb796065ab0404c"}
Nov 03 22:23:15.964925 ip-10-0-143-0 kubenswrapper[2580]: W1103 22:23:15.964900    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d2011b_252e_48d6_be1e_a1a9a8537b39.slice/crio-b1a66291654f731226e1efaa88f001fa2e5b40f455d9b8bd251dae5fdbd58dd9 WatchSource:0}: Error finding container b1a66291654f731226e1efaa88f001fa2e5b40f455d9b8bd251dae5fdbd58dd9: Status 404 returned error can't find the container with id b1a66291654f731226e1efaa88f001fa2e5b40f455d9b8bd251dae5fdbd58dd9
Nov 03 22:23:15.966008 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:15.965944    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"]
Nov 03 22:23:16.038662 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.038627    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"]
Nov 03 22:23:16.042252 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.042223    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.096573 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.096525    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 03 22:23:16.096751 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.096701    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 03 22:23:16.096828 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.096755    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 03 22:23:16.097059 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.097002    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 03 22:23:16.097426 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.097360    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-csvht\""
Nov 03 22:23:16.097754 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.097706    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 03 22:23:16.145714 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.145877 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.145877 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv7fn\" (UniqueName: \"kubernetes.io/projected/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-kube-api-access-kv7fn\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.145877 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-serving-certs-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.145877 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.146034 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.146034 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-metrics-client-ca\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.146034 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.145926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-federate-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.154049 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.150921    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 03 22:23:16.154049 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.151636    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"]
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-metrics-client-ca\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-federate-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv7fn\" (UniqueName: \"kubernetes.io/projected/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-kube-api-access-kv7fn\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.246926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-serving-certs-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.248815 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.247674    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-serving-certs-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.249359 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.249306    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.249950 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.249922    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-metrics-client-ca\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.250899 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.250854    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.251762 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.251737    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-federate-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.251898 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.251877    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.252666 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.252643    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.311697 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.311674    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv7fn\" (UniqueName: \"kubernetes.io/projected/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-kube-api-access-kv7fn\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.348786 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.348757    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:16.358094 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.358069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:16.548765 ip-10-0-143-0 kubenswrapper[2580]: I1103 22:23:16.548737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:16.660533 ip-10-0-143-0 systemd[1]: Stopping Kubernetes Kubelet...
Nov 03 22:23:16.678069 ip-10-0-143-0 systemd[1]: kubelet.service: Deactivated successfully.
Nov 03 22:23:16.678251 ip-10-0-143-0 systemd[1]: Stopped Kubernetes Kubelet.
Nov 03 22:23:16.678438 ip-10-0-143-0 systemd[1]: kubelet.service: Consumed 3.232s CPU time.
Nov 03 22:23:16.694270 ip-10-0-143-0 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 22:23:16.764342 ip-10-0-143-0 kubenswrapper[5162]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:23:16.764342 ip-10-0-143-0 kubenswrapper[5162]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 22:23:16.764342 ip-10-0-143-0 kubenswrapper[5162]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:23:16.764342 ip-10-0-143-0 kubenswrapper[5162]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:23:16.766760 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766674    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.766760 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766755    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.766760 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766761    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.766760 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766765    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766769    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766775    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766778    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766784    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766788    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766792    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766795    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766800    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766803    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766808    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766819    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766823    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766827    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766831    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766835    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766841    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766847    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766852    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766856    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.767027 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766860    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766864    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766868    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766872    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766876    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766880    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766884    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766888    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766892    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766896    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766901    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766908    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766915    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766920    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766926    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766930    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766934    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766939    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766943    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.767929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766948    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766953    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766957    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766962    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766966    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766971    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766976    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766981    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766985    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766989    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766994    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.766998    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767002    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767007    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767011    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767015    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767019    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767023    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767028    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767032    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.768783 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767037    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767041    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767045    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767049    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767054    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767059    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767064    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767068    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767073    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767077    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767081    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767085    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767090    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767094    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767098    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767102    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767107    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767111    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767116    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.769684 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767120    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767124    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767128    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767133    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767137    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767141    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767145    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767150    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767154    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767159    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767163    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767167    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767172    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767176    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767181    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767185    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767189    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767193    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767200    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767204    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.770547 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767208    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767212    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767217    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767221    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.767225    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768564    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768576    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768582    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768588    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768593    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768597    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768603    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768619    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768624    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768628    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768633    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768637    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768641    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768646    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768650    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.771495 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768655    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768659    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768664    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768668    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768672    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768676    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768681    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768685    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768689    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768693    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768698    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768702    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768707    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768711    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768716    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768720    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768724    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768729    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768733    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.772383 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768737    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768742    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768746    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768751    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768755    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768762    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768767    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768772    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768776    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768780    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768784    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768788    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768792    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768796    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768800    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768804    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768808    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768813    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768817    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768829    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.773265 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768834    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768840    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768845    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768851    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768856    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768861    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768866    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768870    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768875    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768879    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768884    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768889    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768893    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768897    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768901    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768905    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768910    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768916    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.773807 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768920    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768932    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768936    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768941    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768945    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768949    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768954    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768958    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768962    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768966    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768971    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768975    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768979    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768983    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768987    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768992    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.768996    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769000    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769004    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769009    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769014    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.774260 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769018    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769022    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769027    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769031    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769056    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769064    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769071    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769076    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769081    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769087    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769093    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769097    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769102    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769903    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769912    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769917    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769922    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769927    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769931    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.774890 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769935    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769940    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769945    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769949    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769953    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769958    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769962    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769966    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769970    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769975    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769979    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769984    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769988    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769992    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.769996    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770001    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770005    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770010    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770014    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770018    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.775359 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770022    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770027    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770031    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770035    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770040    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770043    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770048    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770052    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770056    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770061    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770065    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770069    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770073    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770078    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770089    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770093    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770097    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770101    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770105    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.775864 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770109    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770113    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770118    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770122    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770127    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770132    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770136    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770140    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770144    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770148    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770152    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770156    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770160    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770165    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770169    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770177    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770183    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770188    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770193    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.776315 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770198    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770204    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770209    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770213    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770217    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770221    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770226    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770230    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770233    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770238    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770242    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770247    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770252    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770256    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770261    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770265    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770269    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770273    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770280    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770287    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.776785 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770293    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770298    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770302    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770307    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770311    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770315    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770319    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770325    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770330    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770334    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770339    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770343    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770348    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770352    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770357    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770361    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770365    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770370    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770375    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.777596 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770379    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770383    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.770406    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770540    5162 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770551    5162 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770560    5162 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770566    5162 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770572    5162 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770577    5162 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770585    5162 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770592    5162 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770597    5162 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770602    5162 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770608    5162 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770613    5162 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770618    5162 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770623    5162 flags.go:64] FLAG: --cgroup-root=""
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770627    5162 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770632    5162 flags.go:64] FLAG: --client-ca-file=""
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770637    5162 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770641    5162 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770649    5162 flags.go:64] FLAG: --cluster-domain=""
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770653    5162 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 22:23:16.778333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770658    5162 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770663    5162 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770669    5162 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770675    5162 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770680    5162 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770689    5162 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770694    5162 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770700    5162 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770704    5162 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770709    5162 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770714    5162 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770719    5162 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770725    5162 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770730    5162 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770735    5162 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770740    5162 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770745    5162 flags.go:64] FLAG: --enable-server="true"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770749    5162 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770757    5162 flags.go:64] FLAG: --event-burst="100"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770763    5162 flags.go:64] FLAG: --event-qps="50"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770768    5162 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770773    5162 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770778    5162 flags.go:64] FLAG: --eviction-hard=""
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770784    5162 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 22:23:16.779311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770789    5162 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770794    5162 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770798    5162 flags.go:64] FLAG: --eviction-soft=""
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770803    5162 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770808    5162 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770813    5162 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770818    5162 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770850    5162 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770857    5162 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770862    5162 flags.go:64] FLAG: --feature-gates=""
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770869    5162 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770874    5162 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770878    5162 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770885    5162 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770891    5162 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770897    5162 flags.go:64] FLAG: --help="false"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770901    5162 flags.go:64] FLAG: --hostname-override="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770906    5162 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770911    5162 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770916    5162 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770922    5162 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770927    5162 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770932    5162 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 22:23:16.780638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770937    5162 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770941    5162 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770946    5162 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770953    5162 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770958    5162 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770962    5162 flags.go:64] FLAG: --kube-reserved=""
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770968    5162 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770973    5162 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770978    5162 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770983    5162 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770988    5162 flags.go:64] FLAG: --lock-file=""
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770992    5162 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.770997    5162 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771002    5162 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771011    5162 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771015    5162 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771020    5162 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771025    5162 flags.go:64] FLAG: --logging-format="text"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771030    5162 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771035    5162 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771040    5162 flags.go:64] FLAG: --manifest-url=""
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771044    5162 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771053    5162 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771058    5162 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771073    5162 flags.go:64] FLAG: --max-pods="110"
Nov 03 22:23:16.781696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771078    5162 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771083    5162 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771088    5162 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771093    5162 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771098    5162 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771103    5162 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771107    5162 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771126    5162 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771131    5162 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771137    5162 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771143    5162 flags.go:64] FLAG: --pod-cidr=""
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771148    5162 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771152    5162 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771157    5162 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771161    5162 flags.go:64] FLAG: --port="10250"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771166    5162 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771171    5162 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0f0b51a50cc7d8862"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771176    5162 flags.go:64] FLAG: --qos-reserved=""
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771181    5162 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771186    5162 flags.go:64] FLAG: --register-node="true"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771192    5162 flags.go:64] FLAG: --register-with-taints=""
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771198    5162 flags.go:64] FLAG: --registry-burst="10"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771202    5162 flags.go:64] FLAG: --registry-qps="5"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771207    5162 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771212    5162 flags.go:64] FLAG: --reserved-memory=""
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771217    5162 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 22:23:16.782781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771222    5162 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771227    5162 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771232    5162 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771238    5162 flags.go:64] FLAG: --runonce="false"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771243    5162 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771248    5162 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771254    5162 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771258    5162 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771263    5162 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771268    5162 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771273    5162 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771278    5162 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771283    5162 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771288    5162 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771293    5162 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771297    5162 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771305    5162 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771310    5162 flags.go:64] FLAG: --system-cgroups=""
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771315    5162 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771324    5162 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771328    5162 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771333    5162 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771340    5162 flags.go:64] FLAG: --tls-min-version=""
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771344    5162 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771349    5162 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 22:23:16.783993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771354    5162 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771359    5162 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771364    5162 flags.go:64] FLAG: --v="2"
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771371    5162 flags.go:64] FLAG: --version="false"
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771377    5162 flags.go:64] FLAG: --vmodule=""
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771383    5162 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.771405    5162 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771642    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771650    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771656    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771663    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771668    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771674    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771679    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771684    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771688    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771693    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771698    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771703    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771707    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771711    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.784928 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771716    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771720    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771726    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771730    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771734    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771739    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771743    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771748    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771752    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771756    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771760    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771765    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771769    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771773    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771777    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771782    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771786    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771790    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771795    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771799    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.785565 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771803    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771809    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771813    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771817    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771822    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771826    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771832    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771836    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771841    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771845    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771850    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771854    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771858    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771862    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771868    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771873    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771877    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771881    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771885    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771890    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.786096 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771894    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771898    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771902    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771906    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771911    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771915    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771920    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771924    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771929    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771933    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771937    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771942    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771946    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771951    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771955    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771959    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771964    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771968    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771972    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.786598 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771977    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771981    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771986    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771990    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771995    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.771999    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772003    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772010    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772014    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772018    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772023    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772028    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772034    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772041    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772046    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772051    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772058    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772063    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772069    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.787047 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772075    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772079    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772083    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772088    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772092    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772096    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772101    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772107    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772111    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772116    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772120    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772125    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772129    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772133    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.772174    5162 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772247    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.787526 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772255    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772259    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772264    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772269    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772273    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772281    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772288    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772294    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772299    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772304    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772310    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772315    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772322    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772327    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772332    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772336    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772340    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772345    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.787916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772351    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772355    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772359    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772364    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772368    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772374    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772378    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772383    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772404    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772409    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772413    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772418    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772422    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772427    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772432    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772436    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772440    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772445    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772451    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.788378 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772458    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772462    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772466    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772471    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772475    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772479    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772484    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772489    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772493    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772497    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772502    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772506    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772510    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772515    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772519    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772523    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772527    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772532    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772539    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772543    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.789002 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772547    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772552    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772556    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772560    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772564    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772569    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772573    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772578    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772583    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772587    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772591    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772596    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772601    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772606    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772610    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772614    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772618    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772623    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772627    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772631    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.789500 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772636    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772640    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772644    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772648    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772652    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772657    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772661    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772665    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772670    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772674    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772681    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772685    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772690    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772694    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772698    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772703    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772707    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772712    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772716    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772720    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.789973 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772725    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772729    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772733    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772737    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772743    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772748    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772752    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.772759    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.776986    5162 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.777007    5162 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.777059    5162 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.777067    5162 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777340    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777348    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777354    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777359    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777364    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777369    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777373    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.790487 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777378    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777382    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777422    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777427    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777437    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777441    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777446    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777450    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777455    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777459    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777464    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777468    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777473    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777477    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777508    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777512    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777518    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777527    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777534    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.790967 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777540    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777545    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777550    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777555    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777559    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777563    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777568    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777572    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777576    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777580    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777590    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777594    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777598    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777603    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777607    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777612    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777616    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777621    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777651    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777656    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.791433 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777660    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777664    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777668    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777678    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777682    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777686    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777690    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777694    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777727    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777732    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777736    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777740    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777744    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777751    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777755    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777760    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777768    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777773    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777777    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777782    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.791916 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777788    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777791    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777794    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777797    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777800    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777802    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777807    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777811    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777814    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777817    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777820    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777823    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777828    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777831    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777834    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777837    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777840    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777842    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777845    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.792494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777848    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777851    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777866    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777869    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777872    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777875    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777881    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777884    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777887    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777890    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777892    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777895    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777898    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777901    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777903    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777906    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777909    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777912    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.777917    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778451    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.793162 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778576    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.778632    5162 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778903    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778912    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778917    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778921    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778926    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778931    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778935    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778939    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778944    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778948    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778952    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778958    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778962    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778967    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.793714 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778973    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778977    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778982    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778986    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778991    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778995    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.778999    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779004    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779008    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779013    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779017    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779022    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779026    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779031    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779035    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779039    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779043    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779047    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779052    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.794166 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779056    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779060    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779065    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779069    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779073    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779077    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779082    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779086    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779090    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779094    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779098    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779103    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779107    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779112    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779116    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779121    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779125    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779129    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779136    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.794652 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779143    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779148    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779153    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779157    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779162    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779166    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779171    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779175    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779180    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779184    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779189    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779193    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779198    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779202    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779207    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779211    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779216    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779220    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779224    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.795126 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779230    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779237    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779243    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779247    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779252    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779256    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779261    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779265    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779269    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779273    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779278    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779282    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779287    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779291    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779295    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779300    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779304    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779308    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779313    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779316    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.795720 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779320    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779324    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779328    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779332    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779336    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779340    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779345    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779349    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779357    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779361    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779365    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779370    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779374    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779380    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779385    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.779454    5162 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:23:16.796551 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779506    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779514    5162 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779519    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779525    5162 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779532    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779537    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779541    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779546    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779550    5162 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779556    5162 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779560    5162 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779564    5162 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779569    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779573    5162 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779577    5162 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779582    5162 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779586    5162 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779590    5162 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779594    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:23:16.796964 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779598    5162 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779603    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779607    5162 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779611    5162 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779615    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779621    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779625    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779630    5162 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779634    5162 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779638    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779642    5162 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779648    5162 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779652    5162 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779657    5162 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779661    5162 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779665    5162 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779670    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779674    5162 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779681    5162 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:23:16.797451 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779688    5162 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779693    5162 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779698    5162 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779703    5162 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779708    5162 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779712    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779716    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779721    5162 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779725    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779729    5162 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779733    5162 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779737    5162 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779741    5162 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779745    5162 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779749    5162 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779753    5162 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779757    5162 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779761    5162 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779767    5162 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:23:16.797910 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779771    5162 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779776    5162 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779781    5162 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779785    5162 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779789    5162 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779793    5162 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779798    5162 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779802    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779806    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779811    5162 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779815    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779819    5162 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779823    5162 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779828    5162 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779832    5162 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779836    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779840    5162 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779844    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779848    5162 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779852    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:23:16.798494 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779856    5162 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779860    5162 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779864    5162 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779868    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779872    5162 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779876    5162 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779880    5162 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779884    5162 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779888    5162 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779895    5162 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779899    5162 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779905    5162 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779909    5162 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779914    5162 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779918    5162 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779923    5162 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779927    5162 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779931    5162 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779936    5162 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779940    5162 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:23:16.798985 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779945    5162 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779949    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779953    5162 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779957    5162 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779961    5162 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779966    5162 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779971    5162 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779977    5162 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:16.779982    5162 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.780073    5162 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.781558    5162 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.781619    5162 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.781925    5162 server.go:1014] "Starting client certificate rotation"
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.782055    5162 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.782104    5162 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 22:17:25 +0000 UTC" deadline="2028-05-22 16:57:00.50071321 +0000 UTC"
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.782131    5162 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13578h33m43.718584071s"
Nov 03 22:23:16.799511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.782642    5162 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:23:16.799905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.784276    5162 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:23:16.799905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.785441    5162 log.go:25] "Validated CRI v1 runtime API"
Nov 03 22:23:16.799905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.787136    5162 log.go:25] "Validated CRI v1 image API"
Nov 03 22:23:16.799905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.787875    5162 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 22:23:16.799905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.789778    5162 fs.go:135] Filesystem UUIDs: map[18a61e18-3240-49cd-845c-59f808783e31:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b567883c-693a-4e55-84dc-394080a948b1:/dev/nvme0n1p4]
Nov 03 22:23:16.800064 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.789798    5162 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/211fa1445ea540b35c0fd1a00a4bd1380fd6b777da0dcf430243d785b2577eb1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/211fa1445ea540b35c0fd1a00a4bd1380fd6b777da0dcf430243d785b2577eb1/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/375887f65d35a2142cea7f56c5ceb0aa08b449f9d89b5c9fbd90ec905e3cde14/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/375887f65d35a2142cea7f56c5ceb0aa08b449f9d89b5c9fbd90ec905e3cde14/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3913665de6bc0489d510addc3fb2d23170c23b9c1866e93ef5a410629f17adf8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3913665de6bc0489d510addc3fb2d23170c23b9c1866e93ef5a410629f17adf8/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/45f5199f3ac226370e559e6dadba159fd251f9db973c0e7390aec0ae7fd46ab4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/45f5199f3ac226370e559e6dadba159fd251f9db973c0e7390aec0ae7fd46ab4/userdata/shm major:0 minor:104 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f64aecbb47d76b6ca7409f83c2733481b211c436841337fdf040a28b94b9a47/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f64aecbb47d76b6ca7409f83c2733481b211c436841337fdf040a28b94b9a47/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/666e0b517c01cb2264be5dac231762cfb2aeaedd3e9a956ec726166cfa1b424d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/666e0b517c01cb2264be5dac231762cfb2aeaedd3e9a956ec726166cfa1b424d/userdata/shm major:0 minor:326 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/80f3faaef361a84fd4a85c3449faddada50047c3fbd1a31881369b6ec0f02a9a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/80f3faaef361a84fd4a85c3449faddada50047c3fbd1a31881369b6ec0f02a9a/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82f1234aca9bb72f7064b9e6be00f4b8ed66c8d99535f3864d4160eae1eabf25/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82f1234aca9bb72f7064b9e6be00f4b8ed66c8d99535f3864d4160eae1eabf25/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/835726585c2fb5bdd4dbdbcb5c135f9dd558a4ec18a7a1375b03047a269b94a6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/835726585c2fb5bdd4dbdbcb5c135f9dd558a4ec18a7a1375b03047a269b94a6/userdata/shm major:0 minor:269 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/866f622b8a4b692af18665a9cfcb52bb4555c6473e779a9e952cd7df0ca1edb5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/866f622b8a4b692af18665a9cfcb52bb4555c6473e779a9e952cd7df0ca1edb5/userdata/shm major:0 minor:108 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8be0b40b8c59aa4f24a5ce1151e939106f645a679eaede40c617fe49ddbbf7b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8be0b40b8c59aa4f24a5ce1151e939106f645a679eaede40c617fe49ddbbf7b8/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b1a66291654f731226e1efaa88f001fa2e5b40f455d9b8bd251dae5fdbd58dd9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b1a66291654f731226e1efaa88f001fa2e5b40f455d9b8bd251dae5fdbd58dd9/userdata/shm major:0 minor:354 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b8397cecdf1842716df3cb46db086a53b12abe7166033409494aba2826002d27/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b8397cecdf1842716df3cb46db086a53b12abe7166033409494aba2826002d27/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b998e15949dc74306d9555181b3fd47895f3651206de9bf11f0425acf40cd52a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b998e15949dc74306d9555181b3fd47895f3651206de9bf11f0425acf40cd52a/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bd107d6d9b98acfaaf3eb8889b0e0d8caadf8e5dca98d0328f0598d738c5420c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bd107d6d9b98acfaaf3eb8889b0e0d8caadf8e5dca98d0328f0598d738c5420c/userdata/shm major:0 minor:367 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d17e7ac97721fb5b318182d6bc66b9fe2c0ab35705281414724690cb63d86cbc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d17e7ac97721fb5b318182d6bc66b9fe2c0ab35705281414724690cb63d86cbc/userdata/shm major:0 minor:100 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d57b851a7522e2ec84fc26d236da95894ebf20a5c600959c376029aa87d2f2b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d57b851a7522e2ec84fc26d236da95894ebf20a5c600959c376029aa87d2f2b5/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee289e44e4381a36505fd421ce3b1cbe4cda58125e31c990f3acdc6387faf8da/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee289e44e4381a36505fd421ce3b1cbe4cda58125e31c990f3acdc6387faf8da/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eeb321ea37f35ba1eb4d9f05f6e38c31ff03e2e1fc75c13abdb796065ab0404c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eeb321ea37f35ba1eb4d9f05f6e38c31ff03e2e1fc75c13abdb796065ab0404c/userdata/shm major:0 minor:348 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa9b3c968c9181a97f069d4012e0efdb5493aba639f41f821c82f1be600f1f7c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa9b3c968c9181a97f069d4012e0efdb5493aba639f41f821c82f1be600f1f7c/userdata/shm major:0 minor:266 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~projected/kube-api-access-r8kdc:{mountpoint:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~projected/kube-api-access-r8kdc major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~projected/kube-api-access-6wldl:{mountpoint:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~projected/kube-api-access-6wldl major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:353 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a7f40a7-30f2-48be-a46b-6221ef0b23d5/volumes/kubernetes.io~projected/kube-api-access-brhp9:{mountpoint:/var/lib/kubelet/pods/1a7f40a7-30f2-48be-a46b-6221ef0b23d5/volumes/kubernetes.io~projected/kube-api-access-brhp9 major:0 minor:75 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/248f403e-7506-48d0-9fa2-ed5c621df5fe/volumes/kubernetes.io~projected/kube-api-access-lspjx:{mountpoint:/var/lib/kubelet/pods/248f403e-7506-48d0-9fa2-ed5c621df5fe/volumes/kubernetes.io~projected/kube-api-access-lspjx major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/248f403e-7506-48d0-9fa2-ed5c621df5fe/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/248f403e-7506-48d0-9fa2-ed5c621df5fe/volumes/kubernetes.io~secret/metrics-tls major:0 minor:200 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ebc48f1-c278-4176-9256-5e017ca78b2c/volumes/kubernetes.io~projected/kube-api-access-xfj66:{mountpoint:/var/lib/kubelet/pods/2ebc48f1-c278-4176-9256-5e017ca78b2c/volumes/kubernetes.io~projected/kube-api-access-xfj66 major:0 minor:71 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ebc48f1-c278-4176-9256-5e017ca78b2c/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/2ebc48f1-c278-4176-9256-5e017ca78b2c/volumes/kubernetes.io~secret/metrics-certs major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51712151-3879-46f0-99fb-919861bf9ee5/volumes/kubernetes.io~projected/kube-api-access-4vxhc:{mountpoint:/var/lib/kubelet/pods/51712151-3879-46f0-99fb-919861bf9ee5/volumes/kubernetes.io~projected/kube-api-access-4vxhc major:0 minor:70 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/571f0155-36fa-4eb3-9ddd-9317472e6761/volumes/kubernetes.io~projected/kube-api-access-vp9qw:{mountpoint:/var/lib/kubelet/pods/571f0155-36fa-4eb3-9ddd-9317472e6761/volumes/kubernetes.io~projected/kube-api-access-vp9qw major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~projected/kube-api-access-xcfn9:{mountpoint:/var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~projected/kube-api-access-xcfn9 major:0 minor:263 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~projected/kube-api-access-65p2p:{mountpoint:/var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~projected/kube-api-access-65p2p major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9ae4421-28f8-420f-ade1-c91aafe026ab/volumes/kubernetes.io~projected/kube-api-access-gcjsw:{mountpoint:/var/lib/kubelet/pods/a9ae4421-28f8-420f-ade1-c91aafe026ab/volumes/kubernetes.io~projected/kube-api-access-gcjsw major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9ae4421-28f8-420f-ade1-c91aafe026ab/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/a9ae4421-28f8-420f-ade1-c91aafe026ab/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~projected/kube-api-access-kv7fn:{mountpoint:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~projected/kube-api-access-kv7fn major:0 minor:366 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:68 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~empty-dir/tmp major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~projected/kube-api-access-7s522:{mountpoint:/var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~projected/kube-api-access-7s522 major:0 minor:76 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e7561551-7ec6-43f6-9542-77c479bd3975/volumes/kubernetes.io~projected/kube-api-access-q5hh9:{mountpoint:/var/lib/kubelet/pods/e7561551-7ec6-43f6-9542-77c479bd3975/volumes/kubernetes.io~projected/kube-api-access-q5hh9 major:0 minor:69 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~projected/kube-api-access-6pjm9:{mountpoint:/var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~projected/kube-api-access-6pjm9 major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/efa38b7d-d2f8-439e-88f6-034a12d5b0d1/volumes/kubernetes.io~projected/kube-api-access-phrvx:{mountpoint:/var/lib/kubelet/pods/efa38b7d-d2f8-439e-88f6-034a12d5b0d1/volumes/kubernetes.io~projected/kube-api-access-phrvx major:0 minor:74 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/volumes/kubernetes.io~secret/agent-certs major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f8dae706-de9e-4dc8-868f-961ee5a24428/volumes/kubernetes.io~projected/kube-api-access-5bz7p:{mountpoint:/var/lib/kubelet/pods/f8dae706-de9e-4dc8-868f-961ee5a24428/volumes/kubernetes.io~projected/kube-api-access-5bz7p major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f8dae706-de9e-4dc8-868f-961ee5a24428/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/f8dae706-de9e-4dc8-868f-961ee5a24428/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:196 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volumes/kubernetes.io~projected/kube-api-access-98skz:{mountpoint:/var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volumes/kubernetes.io~projected/kube-api-access-98skz major:0 minor:77 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fa1fb598-13ff-4b53-bec5-85cf10855d09/volumes/kubernetes.io~projected/kube-api-access-fqfx7:{mountpoint:/var/lib/kubelet/pods/fa1fb598-13ff-4b53-bec5-85cf10855d09/volumes/kubernetes.io~projected/kube-api-access-fqfx7 major:0 minor:73 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fba7c426-a1b7-4c37-ba18-8e3fd21dab83/volumes/kubernetes.io~projected/kube-api-access-7hs9m:{mountpoint:/var/lib/kubelet/pods/fba7c426-a1b7-4c37-ba18-8e3fd21dab83/volumes/kubernetes.io~projected/kube-api-access-7hs9m major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fba7c426-a1b7-4c37-ba18-8e3fd21dab83/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/fba7c426-a1b7-4c37-ba18-8e3fd21dab83/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff5c6570-b247-497c-aa51-90e92c38db3c/volumes/kubernetes.io~projected/kube-api-access-8fwjp:{mountpoint:/var/lib/kubelet/pods/ff5c6570-b247-497c-aa51-90e92c38db3c/volumes/kubernetes.io~projected/kube-api-access-8fwjp major:0 minor:72 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/e309d3cc6a8f4d92c869172d2bfc9f88ab44d2f787873beabfecc8fcb35610a9/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/f35078436f6abe61ddfd38b5073c7269e3eb55531fe96edc003229ed92f2278b/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-110:{mountpoint:/var/lib/containers/storage/overlay/88dca930b9a9851f7c6a8557568a737a1b7df2d59c49c10643174463f1fd646c/merged major:0 minor:110 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/634d57466ce921314e5ee8dd2a38365d737c3ddf13df810c647c8bab86d662af/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/f44aa9dab86faa16a2aa432bf38282282db186a8a9993d9e143d33abf0f3124d/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-126:{mountpoint:/var/lib/containers/storage/overlay/a8a7610d7b3ef9f1e27cb63ecc3240bab8c4162afa7f8ca59155de91a71d4ec3/merged major:0 minor:126 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/f01e744dfa8ebfada6877126a192d141afc3968d27d204d5c8d49d924371a50e/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/5ca33a2d0dcaf9ed9a75bbb162b2b6777f7dec6065e9e375b240c50998a02ae2/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/5b3a8d0094f7aa4c5cdb680d2c179c89430b3eee766c2d4467390cec8091a09a/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/37c604293d3ecc619819e27bf835410114e5cf6a6db928af86e54daafe5459de/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/855530fa66156f3f5d4ea7a31718b16670e693979cba608c0cbd64d5540808d3/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-156:{mountpoint:/var/lib/containers/storage/overlay/b39e2db3f3a0c5242fe48b2d5148b1f20ffb31533a18d2f1a572d5022889da39/merged major:0 minor:156 fsType:overlay blockSize:0} overlay_0-161:{mountpoint:/var/lib/containers/storage/overlay/bc50f9d0ebc31e6f1457a548dadfc58c2b10504d55902050404db21551bfd492/merged major:0 minor:161 fsType:overlay blockSize:0} overlay_0-166:{mountpoint:/var/lib/containers/storage/overlay/fe71645cc25ea9af5dc5e737fbe30f089ed3cb45df3d5a3ceca595ca1d426958/merged major:0 minor:166 fsType:overlay blockSize:0} overlay_0-171:{mountpoint:/var/lib/containers/storage/overlay/04fcfaccec6743b3e726df7a7ba1f93c0548d7965097deedaeb7c0859025d09c/merged major:0 minor:171 fsType:overlay blockSize:0} overlay_0-176:{mountpoint:/var/lib/containers/storage/overlay/57fd28e0a5c37ace85aca91e4628adf91763389ea5ac0dc48f0a59c8150f187d/merged major:0 minor:176 fsType:overlay blockSize:0} overlay_0-181:{mountpoint:/var/lib/containers/storage/overlay/a5faf107d9e065ff5e8c681a330280fd6a59f0a455ef9c936038024cf57d40c9/merged major:0 minor:181 fsType:overlay blockSize:0} overlay_0-186:{mountpoint:/var/lib/containers/storage/overlay/d008d245136d604be52b62c754bf79b16e32af4d3174accc7fc7b8cfdc0dc026/merged major:0 minor:186 fsType:overlay blockSize:0} overlay_0-191:{mountpoint:/var/lib/containers/storage/overlay/552273a580446dfc8b9ccdf2f7e33cd11fbb6563727cc3e6939a0973f94f56a7/merged major:0 minor:191 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/be05834da932b830055f8c8b8c41a176f7d34f7427f0d5724871d7980fe31b5d/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/d3f910df1aab85edab45b2eb44f0420ff7a2d123ec3bec94146d50ee4f56bf29/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/2fc0ab58f867c220bae5ec775e010882f9a2ca17b423ed923cd2f07b57e583b2/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/0a8bd5a6fd987b0c9ce1558cdb60e3846fcac8dea90a18ecfc1f17596da4be5f/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/364d783944d08b16c1e90823722b66123e5d568fe53345faa4523f2e08fb46e2/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/cfc5b7d666d626b765d5a85ac33dbb21d02fa971dba50491cf8d0e9a8c690f01/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/7ddfd5006226233493e1cc28ca9b3b71ddb73f97ba3bc83fa2b3505b77026a7c/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/ad056124be135c1be4a8639d5037390510227cc90869a6aeac4281067454eb4e/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/ce5b8b57d95127343e96934c936219f44aed9ac153341e86d121858227742b31/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/212fc425b033a9bf8c95df1b1e4441e5779ab84d22241eed8ef5f49b38d695e1/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/91f396483eea17d98ee0178e6cdd689bba90068d833e7e84d7b0f9841bb3b213/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/dca4602bf1070aa7dba12eea0689ff1865fcde2418f148b1b09c4c8f65c69bc4/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/f0f25587ad5ebf090e8995d2a8bc4831d554090503b18b9e1e5f15181a40f1cc/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/3742dd4ce7f11b4ae10b04ad34b1a3e6cbdb940d95858d3d674bf0a515d092bd/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/111ce5d120ec43d954f17011ebbb94d0dd41bd0a183659e96fa636c0402f47d8/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-283:{mountpoint:/var/lib/containers/storage/overlay/f1f79c8fc72b58e020139e8543ce40057a95bffb46e69b6a73ca282172092a5f/merged major:0 minor:283 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/8e4acff3d5a63a7f376709c3f3df64004e423a0ef44ba43dc1e42756defb1203/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/34c7e46c7ad6ce3f146530f42ca44d39fc2aec983fac685d79473c42b2b67f85/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/9d18098cabf73bee7e8faa317473d241217ca1f64a797d40b5f122c500fdf1e3/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/74b95f8825bf448d577b4c7889e6eda8d8c457ea803586f3c14e9674bda23a86/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/edf0cfb835d97dca4ccba13b244bbc10308c0de6394afcfd70c668e761837a26/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/0cad0a9bcebf16deaa473053996e37ee88d377750780209e75354be9f8eec0ab/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/b8366fb196b4ed990d4a6b2ca66dafbe51d56c36580919b2c3a103c357f21efb/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/bfaa1d27557790cdaa4ef2c8bb893ff92bc9d590c82f37d8ea77c2aaa33b6c29/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-339:{mountpoint:/var/lib/containers/storage/overlay/0e9f06bae11a006b2ab4545ac73b4d2419853eb241ce0270d20535e611449878/merged major:0 minor:339 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/454e357ef834c704198b71c64dc56635d4c955ae492835e5a8b184696156e66e/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/3c8db2742183b52687153540923df2deb17274c75306f44c81437fe482cacaa3/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/8aeecaa3521928739d4eaec7ce959198b1442961951903d1481f8f8f42d8afc2/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/9d3f003a4a35e198fc47cfcbab0b46e7ec1ca9927d2b37a354602911e1de60d6/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/a7797fc7203450baa8ff928801a7c326b1ddaf6ad4db4c4afb07be288b6edf24/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/ac28cc5469199958c2c8e7c084ff5f2124c7c4602ca5d4a7cc841ec0565b0fc7/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-45:{mountpoint:/var/lib/containers/storage/overlay/461ea24b120f138725141bdeb286a11d76424e7aaa4be82a4b1cf2350c326a26/merged major:0 minor:45 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/383b0c1610093a460c94371b0716a865f84cf6d70f77b2f70027621a8ec04b43/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/94daf3cc57efe968c90f77f07078f4afe2aa21977c4f62dc9d5e44cec97cfabd/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-55:{mountpoint:/var/lib/containers/storage/overlay/18985b632506443e3e57f22a8dd832b936c69d7e7490a536ac9193af60c13f6c/merged major:0 minor:55 fsType:overlay blockSize:0} overlay_0-57:{mountpoint:/var/lib/containers/storage/overlay/a5c46e1997e81844a2d5e444b660278cd217cf091a4acad2017a657cb643da6b/merged major:0 minor:57 fsType:overlay blockSize:0} overlay_0-59:{mountpoint:/var/lib/containers/storage/overlay/760b59970adf33f8da024a78b544fb8f4451e723b65ee72c0913bd5981d926c3/merged major:0 minor:59 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/499f94b0b20db65f8f58c69596a6cf67b185b62d3b45ff7d791e634d7c35fbd7/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/2fe68a26f5fd9e2347094d71b5f812ce3a4508abce7a90daf88b004cd3aaa345/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/3eea916a6d54c9e9568e832c32446a7119d56158a598cc1999121e0bd6f4c7c8/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/dd1a174b4503f505999c722a0dfaacade8be9154345ba1791cb90cb649343b38/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/0f34eeb15b077bfc0db485f586be344c9b921fb372309ab9490439ef92ffb35b/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/5904270f0fa52dd28860cae8cb3ebd82fa0233e710b94e54b45471095d0e77a3/merged major:0 minor:98 fsType:overlay blockSize:0}]
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.798571    5162 manager.go:217] Machine: {Timestamp:2026-11-03 22:23:16.798105336 +0000 UTC m=+0.062805338 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105988 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f9b28548d20a661f9a9fb7ca277cc SystemUUID:ec2f9b28-548d-20a6-61f9-a9fb7ca277cc BootID:755477c3-708b-4e1f-8200-8060d5b7d01e Filesystems:[{Device:/var/lib/kubelet/pods/51712151-3879-46f0-99fb-919861bf9ee5/volumes/kubernetes.io~projected/kube-api-access-4vxhc DeviceMajor:0 DeviceMinor:70 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-186 DeviceMajor:0 DeviceMinor:186 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:68 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:217 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d17e7ac97721fb5b318182d6bc66b9fe2c0ab35705281414724690cb63d86cbc/userdata/shm DeviceMajor:0 DeviceMinor:100 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:319 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~projected/kube-api-access-kv7fn DeviceMajor:0 DeviceMinor:366 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/866f622b8a4b692af18665a9cfcb52bb4555c6473e779a9e952cd7df0ca1edb5/userdata/shm DeviceMajor:0 DeviceMinor:108 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-126 DeviceMajor:0 DeviceMinor:126 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-156 DeviceMajor:0 DeviceMinor:156 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-57 DeviceMajor:0 DeviceMinor:57 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ff5c6570-b247-497c-aa51-90e92c38db3c/volumes/kubernetes.io~projected/kube-api-access-8fwjp DeviceMajor:0 DeviceMinor:72 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/835726585c2fb5bdd4dbdbcb5c135f9dd558a4ec18a7a1375b03047a269b94a6/userdata/shm DeviceMajor:0 DeviceMinor:269 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-161 DeviceMajor:0 DeviceMinor:161 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-339 DeviceMajor:0 DeviceMinor:339 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8be0b40b8c59aa4f24a5ce1151e939106f645a679eaede40c617fe49ddbbf7b8/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/211fa1445ea540b35c0fd1a00a4bd1380fd6b777da0dcf430243d785b2577eb1/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-283 DeviceMajor:0 DeviceMinor:283 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/666e0b517c01cb2264be5dac231762cfb2aeaedd3e9a956ec726166cfa1b424d/userdata/shm DeviceMajor:0 DeviceMinor:326 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:360 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/45f5199f3ac226370e559e6dadba159fd251f9db973c0e7390aec0ae7fd46ab4/userdata/shm DeviceMajor:0 DeviceMinor:104 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~projected/kube-api-access-xcfn9 DeviceMajor:0 DeviceMinor:263 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:265 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/248f403e-7506-48d0-9fa2-ed5c621df5fe/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:200 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b8397cecdf1842716df3cb46db086a53b12abe7166033409494aba2826002d27/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82f1234aca9bb72f7064b9e6be00f4b8ed66c8d99535f3864d4160eae1eabf25/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-176 DeviceMajor:0 DeviceMinor:176 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d57b851a7522e2ec84fc26d236da95894ebf20a5c600959c376029aa87d2f2b5/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/80f3faaef361a84fd4a85c3449faddada50047c3fbd1a31881369b6ec0f02a9a/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~projected/kube-api-access-65p2p DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:321 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:361 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-45 DeviceMajor:0 DeviceMinor:45 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f64aecbb47d76b6ca7409f83c2733481b211c436841337fdf040a28b94b9a47/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:264 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e7561551-7ec6-43f6-9542-77c479bd3975/volumes/kubernetes.io~projected/kube-api-access-q5hh9 DeviceMajor:0 DeviceMinor:69 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-55 DeviceMajor:0 DeviceMinor:55 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-59 DeviceMajor:0 DeviceMinor:59 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:315 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:351 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ebc48f1-c278-4176-9256-5e017ca78b2c/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:231 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~projected/kube-api-access-r8kdc DeviceMajor:0 DeviceMinor:327 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a9ae4421-28f8-420f-ade1-c91aafe026ab/volumes/kubernetes.io~projected/kube-api-access-gcjsw DeviceMajor:0 DeviceMinor:347 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-191 DeviceMajor:0 DeviceMinor:191 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volumes/kubernetes.io~projected/kube-api-access-98skz DeviceMajor:0 DeviceMinor:77 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ec301699-cf86-453b-872b-b874de2dcf0d/volumes/kubernetes.io~projected/kube-api-access-6pjm9 DeviceMajor:0 DeviceMinor:268 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fa1fb598-13ff-4b53-bec5-85cf10855d09/volumes/kubernetes.io~projected/kube-api-access-fqfx7 DeviceMajor:0 DeviceMinor:73 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-166 DeviceMajor:0 DeviceMinor:166 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f97dec7e-9808-4b45-8007-fbaacb1f65b5/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/571f0155-36fa-4eb3-9ddd-9317472e6761/volumes/kubernetes.io~projected/kube-api-access-vp9qw DeviceMajor:0 DeviceMinor:232 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:322 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:352 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3913665de6bc0489d510addc3fb2d23170c23b9c1866e93ef5a410629f17adf8/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:262 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/efa38b7d-d2f8-439e-88f6-034a12d5b0d1/volumes/kubernetes.io~projected/kube-api-access-phrvx DeviceMajor:0 DeviceMinor:74 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b998e15949dc74306d9555181b3fd47895f3651206de9bf11f0425acf40cd52a/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~projected/kube-api-access-6wldl DeviceMajor:0 DeviceMinor:350 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:362 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/af060dc0-cbf7-4b75-b3ff-167f74b11ba0/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:363 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee289e44e4381a36505fd421ce3b1cbe4cda58125e31c990f3acdc6387faf8da/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/248f403e-7506-48d0-9fa2-ed5c621df5fe/volumes/kubernetes.io~projected/kube-api-access-lspjx DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fba7c426-a1b7-4c37-ba18-8e3fd21dab83/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8086e88f-4770-416e-93fd-cf324fae5917/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:258 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1695e3d3-d78a-41f2-bbad-67127b64a8fe/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:323 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eeb321ea37f35ba1eb4d9f05f6e38c31ff03e2e1fc75c13abdb796065ab0404c/userdata/shm DeviceMajor:0 DeviceMinor:348 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/16d2011b-252e-48d6-be1e-a1a9a8537b39/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:353 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa9b3c968c9181a97f069d4012e0efdb5493aba639f41f821c82f1be600f1f7c/userdata/shm DeviceMajor:0 DeviceMinor:266 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bd107d6d9b98acfaaf3eb8889b0e0d8caadf8e5dca98d0328f0598d738c5420c/userdata/shm DeviceMajor:0 DeviceMinor:367 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2ebc48f1-c278-4176-9256-5e017ca78b2c/volumes/kubernetes.io~projected/kube-api-access-xfj66 DeviceMajor:0 DeviceMinor:71 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9/volumes/kubernetes.io~projected/kube-api-access-7s522 DeviceMajor:0 DeviceMinor:76 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-171 DeviceMajor:0 DeviceMinor:171 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/375887f65d35a2142cea7f56c5ceb0aa08b449f9d89b5c9fbd90ec905e3cde14/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b1a66291654f731226e1efaa88f001fa2e5b40f455d9b8bd251dae5fdbd58dd9/userdata/shm DeviceMajor:0 DeviceMinor:354 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-110 DeviceMajor:0 DeviceMinor:110 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-181 DeviceMajor:0 DeviceMinor:181 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a9ae4421-28f8-420f-ade1-c91aafe026ab/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f8dae706-de9e-4dc8-868f-961ee5a24428/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:196 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f8dae706-de9e-4dc8-868f-961ee5a24428/volumes/kubernetes.io~projected/kube-api-access-5bz7p DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fba7c426-a1b7-4c37-ba18-8e3fd21dab83/volumes/kubernetes.io~projected/kube-api-access-7hs9m DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a7f40a7-30f2-48be-a46b-6221ef0b23d5/volumes/kubernetes.io~projected/kube-api-access-brhp9 DeviceMajor:0 DeviceMinor:75 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:375887f65d35a21 MacAddress:1e:95:ba:d8:50:44 Speed:10000 Mtu:8901} {Name:3913665de6bc048 MacAddress:b6:de:43:88:23:80 Speed:10000 Mtu:8901} {Name:4f64aecbb47d76b MacAddress:8a:4e:ef:0c:ad:ee Speed:10000 Mtu:8901} {Name:666e0b517c01cb2 MacAddress:52:14:70:18:1b:ae Speed:10000 Mtu:8901} {Name:80f3faaef361a84 MacAddress:26:00:5c:7a:e9:ca Speed:10000 Mtu:8901} {Name:954a5eaa5f345fa MacAddress:6e:27:39:07:69:67 Speed:10000 Mtu:8901} {Name:b1a66291654f731 MacAddress:22:94:b0:18:3b:1f Speed:10000 Mtu:8901} {Name:b8397cecdf18427 MacAddress:5e:df:a0:c8:3d:5e Speed:10000 Mtu:8901} {Name:bd107d6d9b98acf MacAddress:2e:e4:cd:a1:2a:c7 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:df:06:ec:53 Speed:0 Mtu:9001} {Name:br-int MacAddress:f6:e6:3e:20:25:28 Speed:0 Mtu:8901} {Name:d57b851a7522e2e MacAddress:e6:ca:82:ba:07:0f Speed:10000 Mtu:8901} {Name:eeb321ea37f35ba MacAddress:be:f7:6f:4d:fa:58 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:df:06:ec:53 Speed:0 Mtu:9001} {Name:fa9b3c968c9181a MacAddress:7e:be:c7:ff:a2:99 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:86:e8:59:4b:f6:4a Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:32:ed:cc:e4:3d:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799043    5162 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799091    5162 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799441    5162 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799467    5162 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-0.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799606    5162 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799614    5162 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799624    5162 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799646    5162 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799722    5162 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.799897    5162 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.800013    5162 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.800047    5162 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.800058    5162 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.800068    5162 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.800076    5162 kubelet.go:405] "Adding apiserver pod source"
Nov 03 22:23:16.800352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.800084    5162 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 22:23:16.801220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.801200    5162 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 22:23:16.801220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.801219    5162 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:23:16.801729 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.801715    5162 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 22:23:16.807954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.807939    5162 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 22:23:16.807954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.807953    5162 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:23:16.808057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808047    5162 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 03 22:23:16.808232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808221    5162 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 22:23:16.808263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808250    5162 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 22:23:16.808370 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808361    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 22:23:16.808410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808375    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 22:23:16.808410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808381    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 22:23:16.808410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808409    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808416    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808422    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808427    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808433    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808439    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808447    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808455    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808464    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808486    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 22:23:16.808500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.808492    5162 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 22:23:16.811671 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.811657    5162 server.go:1263] "Started kubelet"
Nov 03 22:23:16.811780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.811751    5162 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 22:23:16.811882 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.811735    5162 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 22:23:16.811940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.811893    5162 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 22:23:16.812201 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.812186    5162 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 22:23:16.812260 ip-10-0-143-0 systemd[1]: Started Kubernetes Kubelet.
Nov 03 22:23:16.813461 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.813418    5162 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 22:23:16.816937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.816919    5162 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 22:23:16.817042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.816938    5162 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 22:23:16.817042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.817013    5162 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 22:23:16.817157 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.817061    5162 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 22:23:16.817442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.817285    5162 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 22:23:16.817442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.817027    5162 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 22:17:36 +0000 UTC" deadline="2028-04-12 00:57:39.510228084 +0000 UTC"
Nov 03 22:23:16.819970 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.817324    5162 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12602h34m22.692908865s"
Nov 03 22:23:16.819970 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.819437    5162 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:23:16.820315 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820296    5162 factory.go:55] Registering systemd factory
Nov 03 22:23:16.820423 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820318    5162 factory.go:223] Registration of the systemd container factory successfully
Nov 03 22:23:16.820585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820569    5162 factory.go:153] Registering CRI-O factory
Nov 03 22:23:16.820625 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820589    5162 factory.go:223] Registration of the crio container factory successfully
Nov 03 22:23:16.820661 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820650    5162 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 22:23:16.820723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820671    5162 factory.go:103] Registering Raw factory
Nov 03 22:23:16.820723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820689    5162 manager.go:1199] Started watching for new ooms in manager
Nov 03 22:23:16.820904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820867    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fba7c426-a1b7-4c37-ba18-8e3fd21dab83" volumeName="kubernetes.io/projected/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-kube-api-access-7hs9m" seLinuxMountContext=""
Nov 03 22:23:16.820944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820909    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-grpc-tls" seLinuxMountContext=""
Nov 03 22:23:16.820944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820925    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16d2011b-252e-48d6-be1e-a1a9a8537b39" volumeName="kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-metrics-server-audit-profiles" seLinuxMountContext=""
Nov 03 22:23:16.820944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820938    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51712151-3879-46f0-99fb-919861bf9ee5" volumeName="kubernetes.io/empty-dir/51712151-3879-46f0-99fb-919861bf9ee5-tmp-dir" seLinuxMountContext=""
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820952    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff5c6570-b247-497c-aa51-90e92c38db3c" volumeName="kubernetes.io/configmap/ff5c6570-b247-497c-aa51-90e92c38db3c-serviceca" seLinuxMountContext=""
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820965    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="940f07aa-e2a5-4063-a837-2237c8585bb8" volumeName="kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p" seLinuxMountContext=""
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820978    5162 manager.go:319] Starting recovery of all containers
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820978    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f97dec7e-9808-4b45-8007-fbaacb1f65b5" volumeName="kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-script-lib" seLinuxMountContext=""
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.820990    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ebc48f1-c278-4176-9256-5e017ca78b2c" volumeName="kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs" seLinuxMountContext=""
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821004    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" volumeName="kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821016    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="248f403e-7506-48d0-9fa2-ed5c621df5fe" volumeName="kubernetes.io/projected/248f403e-7506-48d0-9fa2-ed5c621df5fe-kube-api-access-lspjx" seLinuxMountContext=""
Nov 03 22:23:16.821034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821027    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="248f403e-7506-48d0-9fa2-ed5c621df5fe" volumeName="kubernetes.io/secret/248f403e-7506-48d0-9fa2-ed5c621df5fe-metrics-tls" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821038    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821051    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-client-tls" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821074    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821097    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9" volumeName="kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-tmp" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821118    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/projected/1695e3d3-d78a-41f2-bbad-67127b64a8fe-kube-api-access-r8kdc" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821129    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16d2011b-252e-48d6-be1e-a1a9a8537b39" volumeName="kubernetes.io/projected/16d2011b-252e-48d6-be1e-a1a9a8537b39-kube-api-access-6wldl" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821140    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16d2011b-252e-48d6-be1e-a1a9a8537b39" volumeName="kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-tls" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821152    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="248f403e-7506-48d0-9fa2-ed5c621df5fe" volumeName="kubernetes.io/configmap/248f403e-7506-48d0-9fa2-ed5c621df5fe-config-volume" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821163    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44" volumeName="kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821174    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821186    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-tls" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821196    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" volumeName="kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-binary-copy" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821207    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="940f07aa-e2a5-4063-a837-2237c8585bb8" volumeName="kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821219    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ec301699-cf86-453b-872b-b874de2dcf0d" volumeName="kubernetes.io/empty-dir/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-textfile" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821231    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ec301699-cf86-453b-872b-b874de2dcf0d" volumeName="kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821245    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982" volumeName="kubernetes.io/secret/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-agent-certs" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821257    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" volumeName="kubernetes.io/projected/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-kube-api-access-brhp9" seLinuxMountContext=""
Nov 03 22:23:16.821264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821274    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51712151-3879-46f0-99fb-919861bf9ee5" volumeName="kubernetes.io/projected/51712151-3879-46f0-99fb-919861bf9ee5-kube-api-access-4vxhc" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821287    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="940f07aa-e2a5-4063-a837-2237c8585bb8" volumeName="kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821305    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f8dae706-de9e-4dc8-868f-961ee5a24428" volumeName="kubernetes.io/configmap/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821316    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f97dec7e-9808-4b45-8007-fbaacb1f65b5" volumeName="kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-env-overrides" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821327    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ebc48f1-c278-4176-9256-5e017ca78b2c" volumeName="kubernetes.io/projected/2ebc48f1-c278-4176-9256-5e017ca78b2c-kube-api-access-xfj66" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821339    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821350    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16d2011b-252e-48d6-be1e-a1a9a8537b39" volumeName="kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821584    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-federate-client-tls" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821625    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ec301699-cf86-453b-872b-b874de2dcf0d" volumeName="kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821656    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efa38b7d-d2f8-439e-88f6-034a12d5b0d1" volumeName="kubernetes.io/projected/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-kube-api-access-phrvx" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821671    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f8dae706-de9e-4dc8-868f-961ee5a24428" volumeName="kubernetes.io/secret/f8dae706-de9e-4dc8-868f-961ee5a24428-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821688    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="940f07aa-e2a5-4063-a837-2237c8585bb8" volumeName="kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821707    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821723    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f97dec7e-9808-4b45-8007-fbaacb1f65b5" volumeName="kubernetes.io/secret/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821742    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-serving-certs-ca-bundle" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821758    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/projected/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-kube-api-access-kv7fn" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821793    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982" volumeName="kubernetes.io/configmap/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-konnectivity-ca" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821817    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f97dec7e-9808-4b45-8007-fbaacb1f65b5" volumeName="kubernetes.io/projected/f97dec7e-9808-4b45-8007-fbaacb1f65b5-kube-api-access-98skz" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821833    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fba7c426-a1b7-4c37-ba18-8e3fd21dab83" volumeName="kubernetes.io/secret/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-cert" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821857    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff5c6570-b247-497c-aa51-90e92c38db3c" volumeName="kubernetes.io/projected/ff5c6570-b247-497c-aa51-90e92c38db3c-kube-api-access-8fwjp" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821885    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16d2011b-252e-48d6-be1e-a1a9a8537b39" volumeName="kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-client-certs" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821899    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" volumeName="kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821918    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9" volumeName="kubernetes.io/projected/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-kube-api-access-7s522" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821932    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="571f0155-36fa-4eb3-9ddd-9317472e6761" volumeName="kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821955    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7561551-7ec6-43f6-9542-77c479bd3975" volumeName="kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-multus-daemon-config" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821970    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7561551-7ec6-43f6-9542-77c479bd3975" volumeName="kubernetes.io/projected/e7561551-7ec6-43f6-9542-77c479bd3975-kube-api-access-q5hh9" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.821983    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ec301699-cf86-453b-872b-b874de2dcf0d" volumeName="kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-tls" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822003    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f8dae706-de9e-4dc8-868f-961ee5a24428" volumeName="kubernetes.io/projected/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-api-access-5bz7p" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822042    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9" volumeName="kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-tuned" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822063    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/empty-dir/1695e3d3-d78a-41f2-bbad-67127b64a8fe-tmp" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822077    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16d2011b-252e-48d6-be1e-a1a9a8537b39" volumeName="kubernetes.io/empty-dir/16d2011b-252e-48d6-be1e-a1a9a8537b39-audit-log" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822098    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8086e88f-4770-416e-93fd-cf324fae5917" volumeName="kubernetes.io/configmap/8086e88f-4770-416e-93fd-cf324fae5917-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822119    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8086e88f-4770-416e-93fd-cf324fae5917" volumeName="kubernetes.io/projected/8086e88f-4770-416e-93fd-cf324fae5917-kube-api-access-xcfn9" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822133    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="940f07aa-e2a5-4063-a837-2237c8585bb8" volumeName="kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822152    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ec301699-cf86-453b-872b-b874de2dcf0d" volumeName="kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 03 22:23:16.822166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822166    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822188    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8086e88f-4770-416e-93fd-cf324fae5917" volumeName="kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822214    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9ae4421-28f8-420f-ade1-c91aafe026ab" volumeName="kubernetes.io/projected/a9ae4421-28f8-420f-ade1-c91aafe026ab-kube-api-access-gcjsw" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822230    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9ae4421-28f8-420f-ade1-c91aafe026ab" volumeName="kubernetes.io/secret/a9ae4421-28f8-420f-ade1-c91aafe026ab-monitoring-plugin-cert" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822324    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af060dc0-cbf7-4b75-b3ff-167f74b11ba0" volumeName="kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822339    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ec301699-cf86-453b-872b-b874de2dcf0d" volumeName="kubernetes.io/projected/ec301699-cf86-453b-872b-b874de2dcf0d-kube-api-access-6pjm9" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822351    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efa38b7d-d2f8-439e-88f6-034a12d5b0d1" volumeName="kubernetes.io/configmap/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-iptables-alerter-script" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822369    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/configmap/1695e3d3-d78a-41f2-bbad-67127b64a8fe-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822382    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f8dae706-de9e-4dc8-868f-961ee5a24428" volumeName="kubernetes.io/empty-dir/f8dae706-de9e-4dc8-868f-961ee5a24428-data-volume" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822424    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f97dec7e-9808-4b45-8007-fbaacb1f65b5" volumeName="kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-config" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822627    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fa1fb598-13ff-4b53-bec5-85cf10855d09" volumeName="kubernetes.io/projected/fa1fb598-13ff-4b53-bec5-85cf10855d09-kube-api-access-fqfx7" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822667    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695e3d3-d78a-41f2-bbad-67127b64a8fe" volumeName="kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822683    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16d2011b-252e-48d6-be1e-a1a9a8537b39" volumeName="kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-client-ca-bundle" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822697    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="248f403e-7506-48d0-9fa2-ed5c621df5fe" volumeName="kubernetes.io/empty-dir/248f403e-7506-48d0-9fa2-ed5c621df5fe-tmp-dir" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822717    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="940f07aa-e2a5-4063-a837-2237c8585bb8" volumeName="kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822730    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8086e88f-4770-416e-93fd-cf324fae5917" volumeName="kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-tls" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822750    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="940f07aa-e2a5-4063-a837-2237c8585bb8" volumeName="kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822762    5162 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7561551-7ec6-43f6-9542-77c479bd3975" volumeName="kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-cni-binary-copy" seLinuxMountContext=""
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822774    5162 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 22:23:16.823178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.822781    5162 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 22:23:16.845220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.845190    5162 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 22:23:16.846486 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.846466    5162 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 22:23:16.846486 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.846484    5162 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 22:23:16.846640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.846500    5162 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 22:23:16.846640 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.846535    5162 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 22:23:16.860438 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.860373    5162 generic.go:356] "Generic (PLEG): container finished" podID="43c3a967cbec18dc888ae76406901416" containerID="97dbe1ff7ccbcc404d29d98640c63290b94ec8634808b2f5b49138b10aaeafff" exitCode=0
Nov 03 22:23:16.872802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.872776    5162 generic.go:356] "Generic (PLEG): container finished" podID="ec301699-cf86-453b-872b-b874de2dcf0d" containerID="66af4e1d914d581713c078d9e18eb226ce1e9c26b645c2e1cde74e9debfb0f6a" exitCode=0
Nov 03 22:23:16.889823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.889798    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="77c8f1eebd68341660f3081abfb8a2f7f462117657d4269796e43f0f728cbfba" exitCode=0
Nov 03 22:23:16.889823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.889821    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="f261194e9962acc24541eb9fadd74b6f750518b0f1b2408a1033df259aa0a275" exitCode=0
Nov 03 22:23:16.889943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.889829    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="d7d4e4da786900d3f28d36b51ff87d1f5e5ec280b40ad07f7191a511b3657c86" exitCode=0
Nov 03 22:23:16.889943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.889835    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="656bc224758e92f565820c7d72fe573bef3fbb395c8ce9c7dea71a6af6864541" exitCode=0
Nov 03 22:23:16.889943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.889841    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="0dd58c290f34545c64b4a501a35bcf65dca60ae4be0d9ff3338f8886cd73bff6" exitCode=0
Nov 03 22:23:16.889943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.889846    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a7f40a7-30f2-48be-a46b-6221ef0b23d5" containerID="a7f2605858ef405299cdd7a9486c8c8fa72ac0fee2a55680421f62dfe27ecf53" exitCode=0
Nov 03 22:23:16.912295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.912276    5162 manager.go:324] Recovery completed
Nov 03 22:23:16.919905 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.919886    5162 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:23:16.931457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.931438    5162 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:23:16.933148 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.933131    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:23:16.933229 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.933163    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:23:16.933229 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.933179    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:23:16.935791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.935775    5162 cpu_manager.go:226] "Starting" policy="none"
Nov 03 22:23:16.935791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.935787    5162 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 22:23:16.935934 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.935797    5162 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:23:16.935934 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.935803    5162 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 22:23:16.936005 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.935986    5162 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 03 22:23:16.936061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.935996    5162 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 03 22:23:16.936061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936012    5162 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 03 22:23:16.936061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936017    5162 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 03 22:23:16.936061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936022    5162 policy_none.go:50] "Start"
Nov 03 22:23:16.936061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936030    5162 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 22:23:16.936061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936038    5162 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 22:23:16.936237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936153    5162 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 03 22:23:16.936237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936160    5162 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 03 22:23:16.936237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936165    5162 policy_none.go:44] "Start"
Nov 03 22:23:16.936651 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936638    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:23:16.936712 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936668    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:23:16.936712 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.936679    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:23:16.940453 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.940372    5162 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 22:23:16.945899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.945885    5162 manager.go:341] "Starting Device Plugin manager"
Nov 03 22:23:16.945970 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.945909    5162 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 22:23:16.945970 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.945918    5162 server.go:87] "Starting device plugin registration server"
Nov 03 22:23:16.946225 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946212    5162 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 22:23:16.946256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946226    5162 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 22:23:16.946321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946305    5162 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 22:23:16.946589 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946550    5162 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 22:23:16.946678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946599    5162 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 22:23:16.946678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946581    5162 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"]
Nov 03 22:23:16.946783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946696    5162 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:23:16.946783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946605    5162 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 22:23:16.946940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.946790    5162 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T22:23:16.946587769Z","UUID":"38dce877-b5a6-4b62-92cb-84217e78ea89","Handler":null,"Name":"","Endpoint":""}
Nov 03 22:23:16.947374 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.947356    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:23:16.947484 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.947381    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:23:16.947484 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.947409    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:23:16.947717 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.947702    5162 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:23:16.947922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.947907    5162 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:23:16.948030 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.948013    5162 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 22:23:16.948102 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.948065    5162 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-0.us-west-2.compute.internal\" not found"
Nov 03 22:23:16.948465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948448    5162 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 22:23:16.948537 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948472    5162 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 22:23:16.948637 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948619    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:23:16.948702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948646    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:23:16.948702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948652    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:23:16.948702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948676    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:23:16.948702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948691    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:23:16.948877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948657    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:23:16.948986 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.948928    5162 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:16.949110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.948994    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerStarted","Data":"1fc55907adb0a44c70a0a83566482916ca917df7cda700db53aafb1232418414"}
Nov 03 22:23:16.949110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.949035    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerDied","Data":"97dbe1ff7ccbcc404d29d98640c63290b94ec8634808b2f5b49138b10aaeafff"}
Nov 03 22:23:16.949110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.949056    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"43c3a967cbec18dc888ae76406901416","Type":"ContainerStarted","Data":"fb9014ad08f8c58d79900990d29f210191e81184123f021654720c7bc4e46957"}
Nov 03 22:23:16.949282 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.949121    5162 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:23:16.949282 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.949128    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"ffed3ef5033ca77208390bb2ae5eac19","Type":"ContainerStarted","Data":"3b3943e51c8daa17cf549fffd584148f5f90ab86cf8c4be72170ac1425b41055"}
Nov 03 22:23:16.949282 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.949148    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal" event={"ID":"ffed3ef5033ca77208390bb2ae5eac19","Type":"ContainerStarted","Data":"80912b39354056c7346c2ffaa4cf58ade8fc6c30e10a241928d5db845f387d3c"}
Nov 03 22:23:16.949954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.949938    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:23:16.949954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.949958    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:23:16.950089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:16.950023    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:23:16.950185 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:16.950167    5162 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-0.us-west-2.compute.internal\" not found" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.021492 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.021432    5162 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:23:17.021589 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.021554    5162 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 03 22:23:17.022261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.022244    5162 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:23:17.022349 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.022334    5162 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:23:17.046575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.046556    5162 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:23:17.047522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.047502    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:23:17.047576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.047530    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:23:17.047576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.047546    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:23:17.047640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.047585    5162 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.119785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.119760    5162 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.121929 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.121911    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.121986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.121938    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.121986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.121958    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ffed3ef5033ca77208390bb2ae5eac19-config\") pod \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"ffed3ef5033ca77208390bb2ae5eac19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.124553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.124538    5162 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:23:17.178203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.178182    5162 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.178294 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.178239    5162 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.178764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.178751    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeNotReady"
Nov 03 22:23:17.178826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.178778    5162 setters.go:546] "Node became not ready" node="ip-10-0-143-0.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-03T22:23:17Z","lastTransitionTime":"2026-11-03T22:23:17Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 03 22:23:17.184481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.184467    5162 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:23:17.184533 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:17.184504    5162 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.184533 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.184515    5162 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.222407 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.222370    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ffed3ef5033ca77208390bb2ae5eac19-config\") pod \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"ffed3ef5033ca77208390bb2ae5eac19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.222407 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.222409    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.222519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.222426    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.222519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.222487    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ffed3ef5033ca77208390bb2ae5eac19-config\") pod \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"ffed3ef5033ca77208390bb2ae5eac19\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.222596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.222545    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.222596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.222581    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43c3a967cbec18dc888ae76406901416-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" (UID: \"43c3a967cbec18dc888ae76406901416\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.240270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.240248    5162 nodeinfomanager.go:471] existing CSINode "ip-10-0-143-0.us-west-2.compute.internal" is owned by different node (oldNodeID="b16674e7-e1e0-4394-b3f3-b9666b8a551f", newNodeID=""), cleaning up...
Nov 03 22:23:17.286065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.286018    5162 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:23:17.286065 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:17.286055    5162 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.550725 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.550662    5162 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-0.us-west-2.compute.internal" event="NodeReady"
Nov 03 22:23:17.550847 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.550741    5162 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 22:23:17.801173 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.801088    5162 apiserver.go:52] "Watching apiserver"
Nov 03 22:23:17.899906 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.899877    5162 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.900065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.900004    5162 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.960098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.960065    5162 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 22:23:17.960748 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.960721    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-vr4x9","openshift-dns/dns-default-8lb4l","openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k","openshift-image-registry/node-ca-5dtsg","openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc","openshift-ovn-kubernetes/ovnkube-node-lskjf","openshift-multus/multus-6nlml","openshift-multus/multus-additional-cni-plugins-qj4mh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j","openshift-monitoring/thanos-querier-6cc74c986b-hzzr2","openshift-multus/network-metrics-daemon-glv5p","kube-system/konnectivity-agent-d46vm","openshift-ingress-canary/ingress-canary-mr7r8","openshift-network-operator/iptables-alerter-kzxsv","kube-system/global-pull-secret-syncer-86xxh","openshift-insights/insights-runtime-extractor-d5h8h","openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn","openshift-monitoring/node-exporter-jfrn4","openshift-network-diagnostics/network-check-target-cvzp9","openshift-console/console-b495744d6-2wdfb","kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal","openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-5rr7s"]
Nov 03 22:23:17.962652 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.962624    5162 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:23:17.962777 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:17.962694    5162 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.963054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.963020    5162 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:23:17.963129 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:17.963093    5162 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-0.us-west-2.compute.internal"
Nov 03 22:23:17.972847 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:17.972828    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.016472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016442    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lk9bw\""
Nov 03 22:23:18.016695 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016450    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kfchj\""
Nov 03 22:23:18.016695 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016516    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.016695 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016538    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-ljhlr\""
Nov 03 22:23:18.016695 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016548    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.016917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016710    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 22:23:18.016917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016617    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hvj2k\""
Nov 03 22:23:18.016917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016641    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.016917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016563    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.016917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016892    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.016917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016914    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.017197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016963    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.017197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.016978    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 22:23:18.017197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.017106    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.017197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.017149    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.017197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.017172    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 22:23:18.017197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.017175    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.017197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.017189    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nh9f8\""
Nov 03 22:23:18.019370 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.019352    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ztkgs\""
Nov 03 22:23:18.019945 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.019926    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 22:23:18.020048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.019992    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 22:23:18.020048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020004    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 22:23:18.020048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020027    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 22:23:18.020192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020054    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 22:23:18.020192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020066    5162 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 22:23:18.020192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020087    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.020192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020119    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 22:23:18.020192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020130    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.020192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020127    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 22:23:18.020192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020177    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g6sdx\""
Nov 03 22:23:18.020375 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020252    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mdpv5\""
Nov 03 22:23:18.020431 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020413    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 22:23:18.020549 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020525    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 22:23:18.020549 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020539    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hp7n8\""
Nov 03 22:23:18.020695 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020580    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.020695 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020591    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.020825 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020813    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 22:23:18.020859 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.020834    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-kbfvs\""
Nov 03 22:23:18.026049 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026032    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.026102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026059    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.026102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026078    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248f403e-7506-48d0-9fa2-ed5c621df5fe-tmp-dir\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.026192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026100    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.026192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026120    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-system-cni-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.026192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026135    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-log-socket\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.026362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026206    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98skz\" (UniqueName: \"kubernetes.io/projected/f97dec7e-9808-4b45-8007-fbaacb1f65b5-kube-api-access-98skz\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.026362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026243    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248f403e-7506-48d0-9fa2-ed5c621df5fe-metrics-tls\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.026362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026272    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lspjx\" (UniqueName: \"kubernetes.io/projected/248f403e-7506-48d0-9fa2-ed5c621df5fe-kube-api-access-lspjx\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.026362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026299    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.026571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026543    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-run\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.026626 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026593    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248f403e-7506-48d0-9fa2-ed5c621df5fe-tmp-dir\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.026626 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026596    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-kubelet\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.026723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026646    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.026723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026675    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.026723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026686    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-kubelet\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.026824 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026724    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-netns\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.026824 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026755    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-cert\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:18.026824 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026789    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.026824 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026804    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.026964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026826    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8dae706-de9e-4dc8-868f-961ee5a24428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.026964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026905    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-hostroot\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.027022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.026965    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff5c6570-b247-497c-aa51-90e92c38db3c-serviceca\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:23:18.027053 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027023    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-script-lib\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.027053 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027039    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-metrics-client-ca\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.027116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027055    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-netns\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.027116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027082    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-conf-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.027183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027122    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-sys\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.027183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027160    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjsw\" (UniqueName: \"kubernetes.io/projected/a9ae4421-28f8-420f-ade1-c91aafe026ab-kube-api-access-gcjsw\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:18.027268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027188    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.027268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027211    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1695e3d3-d78a-41f2-bbad-67127b64a8fe-tmp\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.027268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027258    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-multus-daemon-config\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.027422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027273    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-script-lib\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.027422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027334    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.027422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027379    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1695e3d3-d78a-41f2-bbad-67127b64a8fe-tmp\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.027580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027439    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.027580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027474    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.027580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027487    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff5c6570-b247-497c-aa51-90e92c38db3c-serviceca\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:23:18.027580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027496    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.027580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027494    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-multus-daemon-config\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.027580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027495    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vxhc\" (UniqueName: \"kubernetes.io/projected/51712151-3879-46f0-99fb-919861bf9ee5-kube-api-access-4vxhc\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:23:18.027580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027561    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027585    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-agent-certs\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027606    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-multus-certs\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027653    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brhp9\" (UniqueName: \"kubernetes.io/projected/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-kube-api-access-brhp9\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027791    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8kdc\" (UniqueName: \"kubernetes.io/projected/1695e3d3-d78a-41f2-bbad-67127b64a8fe-kube-api-access-r8kdc\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027812    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysconfig\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027825    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-agent-certs\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027834    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-federate-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.027877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.027858    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028003    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5hh9\" (UniqueName: \"kubernetes.io/projected/e7561551-7ec6-43f6-9542-77c479bd3975-kube-api-access-q5hh9\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028021    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-root\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028039    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028054    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248f403e-7506-48d0-9fa2-ed5c621df5fe-config-volume\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028068    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-bin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028090    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-client-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028122    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-serving-certs-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028144    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-cnibin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028157    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-original-pull-secret\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.028192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028167    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-conf\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028206    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-client-certs\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028227    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-tls\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028241    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-systemd-units\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028270    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028299    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1695e3d3-d78a-41f2-bbad-67127b64a8fe-metrics-client-ca\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028317    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-host\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028332    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phrvx\" (UniqueName: \"kubernetes.io/projected/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-kube-api-access-phrvx\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028345    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-ovn\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028359    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028439    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028516    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028536    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-kubelet-config\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.028585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028551    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-socket-dir-parent\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028639    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-host-slash\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028657    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-var-lib-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028673    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-env-overrides\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028689    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65p2p\" (UniqueName: \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028713    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028755    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wldl\" (UniqueName: \"kubernetes.io/projected/16d2011b-252e-48d6-be1e-a1a9a8537b39-kube-api-access-6wldl\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028818    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-device-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028844    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51712151-3879-46f0-99fb-919861bf9ee5-tmp-dir\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028861    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028877    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8dae706-de9e-4dc8-868f-961ee5a24428-data-volume\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028908    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-kubernetes\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028924    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51712151-3879-46f0-99fb-919861bf9ee5-tmp-dir\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028933    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-lib-modules\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028943    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-env-overrides\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028964    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8dae706-de9e-4dc8-868f-961ee5a24428-data-volume\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.028993    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff5c6570-b247-497c-aa51-90e92c38db3c-host\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029018    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-metrics-server-audit-profiles\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029071    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-konnectivity-ca\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:23:18.029113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029109    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-registration-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029190    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029214    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029224    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982-konnectivity-ca\") pod \"konnectivity-agent-d46vm\" (UID: \"f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982\") " pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029234    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv7fn\" (UniqueName: \"kubernetes.io/projected/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-kube-api-access-kv7fn\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029262    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-modprobe-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029278    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029293    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-metrics-client-ca\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029317    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-config\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029361    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fwjp\" (UniqueName: \"kubernetes.io/projected/ff5c6570-b247-497c-aa51-90e92c38db3c-kube-api-access-8fwjp\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029409    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029440    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029454    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovnkube-config\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029467    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pjm9\" (UniqueName: \"kubernetes.io/projected/ec301699-cf86-453b-872b-b874de2dcf0d-kube-api-access-6pjm9\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029494    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-iptables-alerter-script\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029520    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029552    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-netd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029581    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cnibin\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029599    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029608    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-etc-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029639    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-sys-fs\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029679    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029706    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029738    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029766    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-multus\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029778    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029791    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-etc-kubernetes\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029823    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bz7p\" (UniqueName: \"kubernetes.io/projected/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-api-access-5bz7p\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029843    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-tuned\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029861    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-systemd\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029885    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-sys\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029909    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-wtmp\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.029922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029934    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-bin\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029958    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-tuned\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.029961    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8dae706-de9e-4dc8-868f-961ee5a24428-crio-socket\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030011    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-systemd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030035    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-node-log\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030062    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030111    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-grpc-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030142    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-var-lib-kubelet\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030170    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9ae4421-28f8-420f-ade1-c91aafe026ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030212    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030242    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-k8s-cni-cncf-io\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030268    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-tmp\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030303    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f97dec7e-9808-4b45-8007-fbaacb1f65b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030323    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-tmp\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030331    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030361    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-socket-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030415    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030441    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hs9m\" (UniqueName: \"kubernetes.io/projected/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-kube-api-access-7hs9m\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030457    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-dbus\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030472    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfn9\" (UniqueName: \"kubernetes.io/projected/8086e88f-4770-416e-93fd-cf324fae5917-kube-api-access-xcfn9\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030488    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqfx7\" (UniqueName: \"kubernetes.io/projected/fa1fb598-13ff-4b53-bec5-85cf10855d09-kube-api-access-fqfx7\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030542    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030569    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-system-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030596    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s522\" (UniqueName: \"kubernetes.io/projected/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-kube-api-access-7s522\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030626    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-textfile\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.030990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.030684    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-textfile\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031095    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-slash\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031115    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51712151-3879-46f0-99fb-919861bf9ee5-hosts-file\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031132    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-cni-binary-copy\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031149    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031168    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031205    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfj66\" (UniqueName: \"kubernetes.io/projected/2ebc48f1-c278-4176-9256-5e017ca78b2c-kube-api-access-xfj66\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031228    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-tls\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031338    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8086e88f-4770-416e-93fd-cf324fae5917-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031367    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7561551-7ec6-43f6-9542-77c479bd3975-cni-binary-copy\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031412    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d2011b-252e-48d6-be1e-a1a9a8537b39-audit-log\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.031644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.031509    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d2011b-252e-48d6-be1e-a1a9a8537b39-audit-log\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.058924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.058874    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2dn77\""
Nov 03 22:23:18.058924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.058915    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 22:23:18.059043 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.058934    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 22:23:18.059126 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.059113    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 22:23:18.059381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.059369    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v9lfz\""
Nov 03 22:23:18.059440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.059381    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kccnl\""
Nov 03 22:23:18.066939 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.066923    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248f403e-7506-48d0-9fa2-ed5c621df5fe-metrics-tls\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.067103 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.067087    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8dae706-de9e-4dc8-868f-961ee5a24428-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.069170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.069157    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebc48f1-c278-4176-9256-5e017ca78b2c-metrics-certs\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:18.089176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.089149    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lspjx\" (UniqueName: \"kubernetes.io/projected/248f403e-7506-48d0-9fa2-ed5c621df5fe-kube-api-access-lspjx\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.089266 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.089180    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brhp9\" (UniqueName: \"kubernetes.io/projected/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-kube-api-access-brhp9\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.089420 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.089381    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98skz\" (UniqueName: \"kubernetes.io/projected/f97dec7e-9808-4b45-8007-fbaacb1f65b5-kube-api-access-98skz\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.089494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.089476    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5hh9\" (UniqueName: \"kubernetes.io/projected/e7561551-7ec6-43f6-9542-77c479bd3975-kube-api-access-q5hh9\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.089534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.089476    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vxhc\" (UniqueName: \"kubernetes.io/projected/51712151-3879-46f0-99fb-919861bf9ee5-kube-api-access-4vxhc\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:23:18.090982 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.090962    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fwjp\" (UniqueName: \"kubernetes.io/projected/ff5c6570-b247-497c-aa51-90e92c38db3c-kube-api-access-8fwjp\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:23:18.092700 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.092682    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp9qw\" (UniqueName: \"kubernetes.io/projected/571f0155-36fa-4eb3-9ddd-9317472e6761-kube-api-access-vp9qw\") pod \"network-check-target-cvzp9\" (UID: \"571f0155-36fa-4eb3-9ddd-9317472e6761\") " pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:18.096761 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.096742    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s522\" (UniqueName: \"kubernetes.io/projected/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-kube-api-access-7s522\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.097774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.097745    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfj66\" (UniqueName: \"kubernetes.io/projected/2ebc48f1-c278-4176-9256-5e017ca78b2c-kube-api-access-xfj66\") pod \"network-metrics-daemon-glv5p\" (UID: \"2ebc48f1-c278-4176-9256-5e017ca78b2c\") " pod="openshift-multus/network-metrics-daemon-glv5p"
Nov 03 22:23:18.098036 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.098012    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqfx7\" (UniqueName: \"kubernetes.io/projected/fa1fb598-13ff-4b53-bec5-85cf10855d09-kube-api-access-fqfx7\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.116611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.116590    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 03 22:23:18.116781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.116762    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.116928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.116910    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 22:23:18.117008 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.116942    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 22:23:18.117008 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.116942    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 22:23:18.117008 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.116966    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.117665 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117645    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 03 22:23:18.117765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117671    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 22:23:18.117765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117690    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.117765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117728    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 22:23:18.117765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117739    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 22:23:18.117765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117742    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-2jqrs\""
Nov 03 22:23:18.117765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117747    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 22:23:18.117765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117758    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vkjzf\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117843    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117924    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.117998    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118003    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118000    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118034    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d9kpm\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118051    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118056    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118090    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 03 22:23:18.118094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118094    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118091    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118116    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118131    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118150    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118152    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vw4kb\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118247    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118286    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118321    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118347    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118349    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-federate-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118377    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118411    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118484    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-client-certs\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118553    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 03 22:23:18.118559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118557    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-57qd7\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118597    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fodrd8fhtnt8d\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118636    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6iiok1m9m0lt2\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118672    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-csvht\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118797    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118829    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118835    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118856    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118898    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118906    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-89fmh\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.118959    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5v4d5\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119004    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119020    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119024    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119057    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c2kdj\""
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119073    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.119213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119071    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119333    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d2011b-252e-48d6-be1e-a1a9a8537b39-metrics-server-audit-profiles\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119496    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-metrics-client-ca\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119527    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119557    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8kdc\" (UniqueName: \"kubernetes.io/projected/1695e3d3-d78a-41f2-bbad-67127b64a8fe-kube-api-access-r8kdc\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119565    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119627    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119698    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjsw\" (UniqueName: \"kubernetes.io/projected/a9ae4421-28f8-420f-ade1-c91aafe026ab-kube-api-access-gcjsw\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119743    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-iptables-alerter-script\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119912    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.119961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.119955    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-secret-telemeter-client\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.120337 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.120314    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 22:23:18.120472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.120452    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a9ae4421-28f8-420f-ade1-c91aafe026ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdkbc\" (UID: \"a9ae4421-28f8-420f-ade1-c91aafe026ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:18.120556 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.120467    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-grpc-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.121134 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.121118    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-tls\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.121134 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.121122    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.122371 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.122350    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.122371 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.122367    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-tls\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.122501 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.122398    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8086e88f-4770-416e-93fd-cf324fae5917-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.122501 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.122378    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.125110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.124983    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bz7p\" (UniqueName: \"kubernetes.io/projected/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-api-access-5bz7p\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.125110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.125012    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pjm9\" (UniqueName: \"kubernetes.io/projected/ec301699-cf86-453b-872b-b874de2dcf0d-kube-api-access-6pjm9\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.125110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.125059    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrvx\" (UniqueName: \"kubernetes.io/projected/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-kube-api-access-phrvx\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:23:18.125304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.125225    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv7fn\" (UniqueName: \"kubernetes.io/projected/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-kube-api-access-kv7fn\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.125304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.125239    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65p2p\" (UniqueName: \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.125304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.125245    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hs9m\" (UniqueName: \"kubernetes.io/projected/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-kube-api-access-7hs9m\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:18.125304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.125248    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcfn9\" (UniqueName: \"kubernetes.io/projected/8086e88f-4770-416e-93fd-cf324fae5917-kube-api-access-xcfn9\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.125603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.125588    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wldl\" (UniqueName: \"kubernetes.io/projected/16d2011b-252e-48d6-be1e-a1a9a8537b39-kube-api-access-6wldl\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.126539 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.126517    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1695e3d3-d78a-41f2-bbad-67127b64a8fe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.126675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.126659    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8dae706-de9e-4dc8-868f-961ee5a24428-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.126719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.126682    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 03 22:23:18.127802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.127779    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.127802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.127794    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8086e88f-4770-416e-93fd-cf324fae5917-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-rs9pf\" (UID: \"8086e88f-4770-416e-93fd-cf324fae5917\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-rs9pf"
Nov 03 22:23:18.127914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.127812    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.127914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.127826    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba7c426-a1b7-4c37-ba18-8e3fd21dab83-cert\") pod \"ingress-canary-mr7r8\" (UID: \"fba7c426-a1b7-4c37-ba18-8e3fd21dab83\") " pod="openshift-ingress-canary/ingress-canary-mr7r8"
Nov 03 22:23:18.127914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.127797    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\") pod \"console-b495744d6-2wdfb\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") " pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:18.127914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.127857    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-metrics-client-ca\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.129045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.129022    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-secret-metrics-server-tls\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.129045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.129030    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-client-tls\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.129182 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.129108    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1695e3d3-d78a-41f2-bbad-67127b64a8fe-metrics-client-ca\") pod \"thanos-querier-6cc74c986b-hzzr2\" (UID: \"1695e3d3-d78a-41f2-bbad-67127b64a8fe\") " pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:18.129182 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.129113    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-serving-certs-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.129182 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.129130    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248f403e-7506-48d0-9fa2-ed5c621df5fe-config-volume\") pod \"dns-default-8lb4l\" (UID: \"248f403e-7506-48d0-9fa2-ed5c621df5fe\") " pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:18.129182 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.129169    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d2011b-252e-48d6-be1e-a1a9a8537b39-client-ca-bundle\") pod \"metrics-server-5bbc88ffc8-7mgdn\" (UID: \"16d2011b-252e-48d6-be1e-a1a9a8537b39\") " pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:18.132467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132447    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.132522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132481    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-k8s-cni-cncf-io\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.132522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132506    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.132581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132531    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-socket-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.132581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132553    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-k8s-cni-cncf-io\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.132581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132555    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132556    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-dbus\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132585    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132611    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-system-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132623    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-socket-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132631    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-slash\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132613    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-dbus\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132652    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51712151-3879-46f0-99fb-919861bf9ee5-hosts-file\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132675    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-system-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132678    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-system-cni-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.132697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132697    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-system-cni-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132699    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-log-socket\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132736    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51712151-3879-46f0-99fb-919861bf9ee5-hosts-file\") pod \"node-resolver-vr4x9\" (UID: \"51712151-3879-46f0-99fb-919861bf9ee5\") " pod="openshift-dns/node-resolver-vr4x9"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132716    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-log-socket\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132693    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-slash\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132749    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-run\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132773    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-kubelet\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132793    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-run\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132807    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132824    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-kubelet\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132831    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-kubelet\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132852    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-netns\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132898    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-hostroot\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132908    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-cni-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132917    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-run-netns\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132910    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-kubelet\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132943    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-netns\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132960    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-hostroot\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132967    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-conf-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.132987    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-sys\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133017    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-sys\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133024    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-netns\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133026    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-conf-dir\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133065    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.133097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133094    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133117    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-multus-certs\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133142    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysconfig\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133147    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133166    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-root\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133179    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133182    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysconfig\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133184    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-run-multus-certs\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133192    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-bin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133204    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-root\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133216    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-cnibin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133225    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-bin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133241    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-conf\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133267    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-systemd-units\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133284    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-cnibin\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133300    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133308    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-systemd-units\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133324    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-host\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133345    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-ovn\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133360    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133367    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-kubelet-config\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133372    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-host\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133412    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-conf\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133426    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-socket-dir-parent\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133448    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-host-slash\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133462    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-var-lib-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133478    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-device-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133433    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44-kubelet-config\") pod \"global-pull-secret-syncer-86xxh\" (UID: \"48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44\") " pod="kube-system/global-pull-secret-syncer-86xxh"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133501    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-kubernetes\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133430    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-ovn\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133503    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efa38b7d-d2f8-439e-88f6-034a12d5b0d1-host-slash\") pod \"iptables-alerter-kzxsv\" (UID: \"efa38b7d-d2f8-439e-88f6-034a12d5b0d1\") " pod="openshift-network-operator/iptables-alerter-kzxsv"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133525    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-lib-modules\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133538    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-device-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133547    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff5c6570-b247-497c-aa51-90e92c38db3c-host\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133571    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-registration-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133578    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-multus-socket-dir-parent\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133537    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-var-lib-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133578    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-kubernetes\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133592    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-modprobe-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133620    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff5c6570-b247-497c-aa51-90e92c38db3c-host\") pod \"node-ca-5dtsg\" (UID: \"ff5c6570-b247-497c-aa51-90e92c38db3c\") " pod="openshift-image-registry/node-ca-5dtsg"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133640    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-lib-modules\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133654    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-registration-dir\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133683    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133740    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-modprobe-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133763    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-sysctl-d\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133778    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133803    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-netd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133825    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133825    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cnibin\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133852    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-netd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133854    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-etc-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133875    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a7f40a7-30f2-48be-a46b-6221ef0b23d5-cnibin\") pod \"multus-additional-cni-plugins-qj4mh\" (UID: \"1a7f40a7-30f2-48be-a46b-6221ef0b23d5\") " pod="openshift-multus/multus-additional-cni-plugins-qj4mh"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133876    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-sys-fs\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133905    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-multus\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133909    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-etc-openvswitch\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133929    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-etc-kubernetes\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133968    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa1fb598-13ff-4b53-bec5-85cf10855d09-sys-fs\") pod \"aws-ebs-csi-driver-node-rcs6j\" (UID: \"fa1fb598-13ff-4b53-bec5-85cf10855d09\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcs6j"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133972    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-etc-kubernetes\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133969    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7561551-7ec6-43f6-9542-77c479bd3975-host-var-lib-cni-multus\") pod \"multus-6nlml\" (UID: \"e7561551-7ec6-43f6-9542-77c479bd3975\") " pod="openshift-multus/multus-6nlml"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.133971    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-systemd\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134013    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-etc-systemd\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134014    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-sys\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134046    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-sys\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134056    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-wtmp\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134086    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-bin\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.134065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134110    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8dae706-de9e-4dc8-868f-961ee5a24428-crio-socket\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134131    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec301699-cf86-453b-872b-b874de2dcf0d-node-exporter-wtmp\") pod \"node-exporter-jfrn4\" (UID: \"ec301699-cf86-453b-872b-b874de2dcf0d\") " pod="openshift-monitoring/node-exporter-jfrn4"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134131    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-systemd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134155    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-node-log\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134160    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-run-systemd\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134170    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-var-lib-kubelet\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134159    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8dae706-de9e-4dc8-868f-961ee5a24428-crio-socket\") pod \"insights-runtime-extractor-d5h8h\" (UID: \"f8dae706-de9e-4dc8-868f-961ee5a24428\") " pod="openshift-insights/insights-runtime-extractor-d5h8h"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134153    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-host-cni-bin\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134180    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f97dec7e-9808-4b45-8007-fbaacb1f65b5-node-log\") pod \"ovnkube-node-lskjf\" (UID: \"f97dec7e-9808-4b45-8007-fbaacb1f65b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:18.136193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.134225    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9-var-lib-kubelet\") pod \"tuned-5rr7s\" (UID: \"b3055bb4-b2a7-4ae2-9f54-bc4f4a21a1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-5rr7s"
Nov 03 22:23:18.136608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.136592    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af060dc0-cbf7-4b75-b3ff-167f74b11ba0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74dbbb49b8-2nl5k\" (UID: \"af060dc0-cbf7-4b75-b3ff-167f74b11ba0\") " pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.272887 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.272867    5162 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:23:18.273302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.273281    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k"
Nov 03 22:23:18.352035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.351974    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:18.603086 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.603009    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:18.688195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.688168    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-655dc496c7-lzcqw"]
Nov 03 22:23:18.690452 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.690436    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738515    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-oauth-config\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738557    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-trusted-ca-bundle\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738575    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-oauth-serving-cert\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738727    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-service-ca\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738838 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738781    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-config\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738838 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738816    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490753a-2874-4ba6-9fb3-fa9fce802e9e-tmp\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738899    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-serving-cert\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.738957 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.738925    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c968b\" (UniqueName: \"kubernetes.io/projected/b490753a-2874-4ba6-9fb3-fa9fce802e9e-kube-api-access-c968b\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.745504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.745477    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 03 22:23:18.756694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.756674    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-655dc496c7-lzcqw"]
Nov 03 22:23:18.840213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840183    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-oauth-config\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840215    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-trusted-ca-bundle\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840230    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-oauth-serving-cert\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840262    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-service-ca\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840299    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-config\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840315    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490753a-2874-4ba6-9fb3-fa9fce802e9e-tmp\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840343    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-serving-cert\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840406    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c968b\" (UniqueName: \"kubernetes.io/projected/b490753a-2874-4ba6-9fb3-fa9fce802e9e-kube-api-access-c968b\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.840682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840619    5162 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 22:23:18.841022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.840899    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490753a-2874-4ba6-9fb3-fa9fce802e9e-tmp\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.841117 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.841099    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-config\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.841199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.841175    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-oauth-serving-cert\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.841322 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.841302    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-trusted-ca-bundle\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.841383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.841344    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-service-ca\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.842830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.842809    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-oauth-config\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.842997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.842978    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-serving-cert\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.904997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.904943    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c968b\" (UniqueName: \"kubernetes.io/projected/b490753a-2874-4ba6-9fb3-fa9fce802e9e-kube-api-access-c968b\") pod \"console-655dc496c7-lzcqw\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") " pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:18.907232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:18.907204    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k" event={"ID":"af060dc0-cbf7-4b75-b3ff-167f74b11ba0","Type":"ContainerStarted","Data":"bd107d6d9b98acfaaf3eb8889b0e0d8caadf8e5dca98d0328f0598d738c5420c"}
Nov 03 22:23:19.000832 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.000807    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:19.212595 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.212572    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:19.369868 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.369838    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-655dc496c7-lzcqw"]
Nov 03 22:23:19.415835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.415798    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:23:19.419942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.419798    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.444997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.444972    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445633    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445673    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445702    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-config\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445726    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445755    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445781    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445808    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445835    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445871    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445900    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445936    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445964    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.445987    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.446023    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.446046    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.446075    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.446129    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.446159    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.446196    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.446355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.446218    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9q7f\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-kube-api-access-w9q7f\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.480356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.480291    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 03 22:23:19.480641 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.480475    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42qr9\""
Nov 03 22:23:19.491422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.488632    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:23:19.491422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.489088    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 03 22:23:19.491422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.489248    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 03 22:23:19.491422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.489291    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 03 22:23:19.491422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.490949    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 03 22:23:19.491422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.491225    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547263    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547297    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547319    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547334    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9q7f\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-kube-api-access-w9q7f\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547357    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547376    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547413    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-config\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547436    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547465    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547490    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547516    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547539    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547580    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547619    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547654    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547682    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547705    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547741    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547766    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547794    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.547868    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.548264    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.548408    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.549293    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.549407    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.549767    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.549842    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.551448    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.552522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.552100    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.554583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.554541    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.580660 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.580510    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 03 22:23:19.580660 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.580577    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 03 22:23:19.580660 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.580513    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 03 22:23:19.580916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.580739    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 03 22:23:19.580916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.580776    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 03 22:23:19.581356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.581109    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 03 22:23:19.581356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.581208    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7h3uqh5pcgour\""
Nov 03 22:23:19.581356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.581275    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 03 22:23:19.581356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.581329    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 03 22:23:19.591804 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.591776    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.592034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.592013    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.593534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.592359    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.593534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.592679    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.593534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.592876    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-config\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.593534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.593083    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.594324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.593818    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.594324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.594291    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.594735 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.594524    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.611768 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.611744    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9q7f\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-kube-api-access-w9q7f\") pod \"prometheus-k8s-0\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.776356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.776279    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:19.915263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.915232    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-655dc496c7-lzcqw" event={"ID":"b490753a-2874-4ba6-9fb3-fa9fce802e9e","Type":"ContainerStarted","Data":"d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1"}
Nov 03 22:23:19.915602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:19.915291    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-655dc496c7-lzcqw" event={"ID":"b490753a-2874-4ba6-9fb3-fa9fce802e9e","Type":"ContainerStarted","Data":"575173b253283ec57d89d80ded4f78533b6def5b3bec4cd471243dc567a59603"}
Nov 03 22:23:20.174127 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.173183    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:20.175966 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.175937    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8lb4l"
Nov 03 22:23:20.202157 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.202134    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:23:20.206623 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:20.206592    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e40242d_93d1_4ee5_8028_f180905714d8.slice/crio-edc2d6af90269ad11493e4dc4531babc108dce8831be45a50097363b43c988a4 WatchSource:0}: Error finding container edc2d6af90269ad11493e4dc4531babc108dce8831be45a50097363b43c988a4: Status 404 returned error can't find the container with id edc2d6af90269ad11493e4dc4531babc108dce8831be45a50097363b43c988a4
Nov 03 22:23:20.345157 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.345047    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-655dc496c7-lzcqw" podStartSLOduration=2.345031884 podStartE2EDuration="2.345031884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:20.209784601 +0000 UTC m=+3.474484639" watchObservedRunningTime="2026-11-03 22:23:20.345031884 +0000 UTC m=+3.609731901"
Nov 03 22:23:20.568241 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.568202    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:20.827553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.827520    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:20.923145 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.923111    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2" event={"ID":"1695e3d3-d78a-41f2-bbad-67127b64a8fe","Type":"ContainerStarted","Data":"1c94e5bc7af92dee62c27ab846d38660252f567f85d8052e20e35643135aa6f9"}
Nov 03 22:23:20.923630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.923188    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2" event={"ID":"1695e3d3-d78a-41f2-bbad-67127b64a8fe","Type":"ContainerStarted","Data":"88cbe977841dfec56279edda003c0a1142b52feaa23a1c98ddde899688779f5d"}
Nov 03 22:23:20.923630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.923231    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2" event={"ID":"1695e3d3-d78a-41f2-bbad-67127b64a8fe","Type":"ContainerStarted","Data":"755d28fadf05daf15606ba7156d173ff9e8b18243a624c38b7b896e9dd10c5e8"}
Nov 03 22:23:20.924610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.924577    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerStarted","Data":"edc2d6af90269ad11493e4dc4531babc108dce8831be45a50097363b43c988a4"}
Nov 03 22:23:20.983726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.983696    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:20.988174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:20.988150    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:21.040578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.040547    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:21.192755 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.192731    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:21.192912 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.192901    5162 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 22:23:21.261747 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.261706    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:21.475881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.475800    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:21.678213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.678175    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:21.929108 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.929080    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn" event={"ID":"16d2011b-252e-48d6-be1e-a1a9a8537b39","Type":"ContainerStarted","Data":"56b7c8e9821d77742de801a354809c5dfe525d601bdece2b843c48d7cc6c83ee"}
Nov 03 22:23:21.930571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.930549    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc" event={"ID":"a9ae4421-28f8-420f-ade1-c91aafe026ab","Type":"ContainerStarted","Data":"3cdfbf4bfe6909250572594fb2d69a01cedc7fb99f337c90e68973d434956d33"}
Nov 03 22:23:21.932172 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.932148    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k" event={"ID":"af060dc0-cbf7-4b75-b3ff-167f74b11ba0","Type":"ContainerStarted","Data":"f2afdea7471a95b6fc82b0ac59c1a9bb1d522b4c5bc90b2b1cc2023a2b3ea6b2"}
Nov 03 22:23:21.939017 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:21.939001    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6cc74c986b-hzzr2"
Nov 03 22:23:22.320672 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:22.320598    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:22.568520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:22.568492    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:22.937879 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:22.937849    5162 generic.go:356] "Generic (PLEG): container finished" podID="9e40242d-93d1-4ee5-8028-f180905714d8" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124" exitCode=0
Nov 03 22:23:22.938301 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:22.937938    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"}
Nov 03 22:23:23.668266 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:23.668229    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:23.874770 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:23.874734    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:23.950400 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:23.950311    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k" event={"ID":"af060dc0-cbf7-4b75-b3ff-167f74b11ba0","Type":"ContainerStarted","Data":"5f56ef5cb64cfe4124d64fb501713187bae378aa6d608b07cfcc03cfad429f09"}
Nov 03 22:23:23.950755 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:23.950406    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74dbbb49b8-2nl5k" event={"ID":"af060dc0-cbf7-4b75-b3ff-167f74b11ba0","Type":"ContainerStarted","Data":"0c4a0c95508780b1a47a60fc0d4f6856b14aedc403eec14fc888ebf298c14b6a"}
Nov 03 22:23:24.110755 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:24.110728    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lskjf"
Nov 03 22:23:24.117558 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:24.117532    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:24.122976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:24.122952    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:24.430421 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:24.430365    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:24.634552 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:24.634517    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:24.958620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:24.958588    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:23:25.106724 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:25.106685    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b495744d6-2wdfb"]
Nov 03 22:23:25.182304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:25.182265    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:25.442807 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:25.442779    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:26.250313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:26.250278    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:26.498776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:26.498745    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:26.964529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:26.964500    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerStarted","Data":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"}
Nov 03 22:23:26.964642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:26.964570    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerStarted","Data":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"}
Nov 03 22:23:26.964642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:26.964597    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerStarted","Data":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"}
Nov 03 22:23:26.964642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:26.964635    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerStarted","Data":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"}
Nov 03 22:23:27.286848 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.286766    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-94b457789-fvcgc"]
Nov 03 22:23:27.290575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.290557    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.317874 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.317845    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:27.317986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.317968    5162 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 22:23:27.322942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.322922    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdkbc"
Nov 03 22:23:27.384656 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.384626    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94b457789-fvcgc"]
Nov 03 22:23:27.420235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420204    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-serving-cert\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.420382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420253    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-oauth-serving-cert\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.420382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420360    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-config\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.420522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420422    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-trusted-ca-bundle\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.420522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420455    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-oauth-config\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.420621 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420594    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-service-ca\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.420677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420633    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6296a6f9-642b-44c3-8715-1b925c6bf1bf-tmp\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.420737 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.420702    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmn8p\" (UniqueName: \"kubernetes.io/projected/6296a6f9-642b-44c3-8715-1b925c6bf1bf-kube-api-access-dmn8p\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521286 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521259    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-serving-cert\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521297    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-oauth-serving-cert\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521318    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-config\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521335    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-trusted-ca-bundle\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521451    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-oauth-config\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521507    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-service-ca\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521542    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6296a6f9-642b-44c3-8715-1b925c6bf1bf-tmp\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.521723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.521593    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmn8p\" (UniqueName: \"kubernetes.io/projected/6296a6f9-642b-44c3-8715-1b925c6bf1bf-kube-api-access-dmn8p\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.522121 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.522044    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6296a6f9-642b-44c3-8715-1b925c6bf1bf-tmp\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.522250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.522154    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-oauth-serving-cert\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.522250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.522154    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-config\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.522436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.522411    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-service-ca\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.522568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.522548    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-trusted-ca-bundle\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.523867 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.523842    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-oauth-config\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.524082 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.524065    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-serving-cert\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.584293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.584222    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmn8p\" (UniqueName: \"kubernetes.io/projected/6296a6f9-642b-44c3-8715-1b925c6bf1bf-kube-api-access-dmn8p\") pod \"console-94b457789-fvcgc\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") " pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.601206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.601186    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:27.744430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.744378    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:27.972151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.972110    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerStarted","Data":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"}
Nov 03 22:23:27.972290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.972196    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerStarted","Data":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"}
Nov 03 22:23:27.986752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:27.986724    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:28.014101 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:23:28.014060    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6296a6f9_642b_44c3_8715_1b925c6bf1bf.slice/crio-4419ca58dc1a8afa9ee0f876512cc4cfcd5d529656c47ea334386c91ddc60da3 WatchSource:0}: Error finding container 4419ca58dc1a8afa9ee0f876512cc4cfcd5d529656c47ea334386c91ddc60da3: Status 404 returned error can't find the container with id 4419ca58dc1a8afa9ee0f876512cc4cfcd5d529656c47ea334386c91ddc60da3
Nov 03 22:23:28.015415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:28.015369    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94b457789-fvcgc"]
Nov 03 22:23:28.108819 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:28.108759    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.008110634 podStartE2EDuration="9.108740655s" totalImagesPullingTime="6.077919218s" totalInitContainerRuntime="22.710803ms" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:28.107545417 +0000 UTC m=+11.372245428" watchObservedRunningTime="2026-11-03 22:23:28.108740655 +0000 UTC m=+11.373440667"
Nov 03 22:23:28.153578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:28.153547    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:28.669526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:28.669490    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:28.872876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:28.872842    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:28.977976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:28.977898    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94b457789-fvcgc" event={"ID":"6296a6f9-642b-44c3-8715-1b925c6bf1bf","Type":"ContainerStarted","Data":"cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24"}
Nov 03 22:23:28.977976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:28.977954    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94b457789-fvcgc" event={"ID":"6296a6f9-642b-44c3-8715-1b925c6bf1bf","Type":"ContainerStarted","Data":"4419ca58dc1a8afa9ee0f876512cc4cfcd5d529656c47ea334386c91ddc60da3"}
Nov 03 22:23:29.105813 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:29.105755    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-94b457789-fvcgc" podStartSLOduration=2.105743137 podStartE2EDuration="2.105743137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:29.10546584 +0000 UTC m=+12.370165851" watchObservedRunningTime="2026-11-03 22:23:29.105743137 +0000 UTC m=+12.370443148"
Nov 03 22:23:29.195355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:29.195327    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:29.389429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:29.389337    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:30.531492 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:30.531456    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:33.312221 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:33.312180    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:33.511573 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:33.511534    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:34.177310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:34.177282    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:34.387485 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:34.387453    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:36.639904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:36.639874    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d46vm"
Nov 03 22:23:36.718433 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:36.718381    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:36.923841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:36.923812    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:37.601878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:37.601838    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:37.602029 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:37.601898    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:37.606442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:37.606420    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:38.019286 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:38.019260    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:23:38.170661 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:38.170630    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-655dc496c7-lzcqw"]
Nov 03 22:23:39.537015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:39.536976    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:39.740014 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:39.739980    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:40.083848 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:40.083815    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:23:41.446878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:41.446843    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:41.451901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:41.451879    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:42.035674 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:42.035647    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5bbc88ffc8-7mgdn"
Nov 03 22:23:45.185282 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:45.185250    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cvzp9"
Nov 03 22:23:47.147118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:47.147079    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:47.399443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:47.399340    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:48.746164 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:48.746125    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:48.803888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:48.803865    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:49.074613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:49.074546    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:23:50.129490 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.129432    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-b495744d6-2wdfb" podUID="940f07aa-e2a5-4063-a837-2237c8585bb8" containerName="console" containerID="cri-o://f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2" gracePeriod=15
Nov 03 22:23:50.415483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.415457    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:50.529072 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529045    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\") pod \"940f07aa-e2a5-4063-a837-2237c8585bb8\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") "
Nov 03 22:23:50.529217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529109    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\") pod \"940f07aa-e2a5-4063-a837-2237c8585bb8\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") "
Nov 03 22:23:50.529217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529139    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\" (UniqueName: \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\") pod \"940f07aa-e2a5-4063-a837-2237c8585bb8\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") "
Nov 03 22:23:50.529306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529231    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\") pod \"940f07aa-e2a5-4063-a837-2237c8585bb8\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") "
Nov 03 22:23:50.529306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529268    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\" (UniqueName: \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\") pod \"940f07aa-e2a5-4063-a837-2237c8585bb8\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") "
Nov 03 22:23:50.529421 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529319    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\") pod \"940f07aa-e2a5-4063-a837-2237c8585bb8\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") "
Nov 03 22:23:50.529421 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529340    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\") pod \"940f07aa-e2a5-4063-a837-2237c8585bb8\" (UID: \"940f07aa-e2a5-4063-a837-2237c8585bb8\") "
Nov 03 22:23:50.529639 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529614    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config" pod "940f07aa-e2a5-4063-a837-2237c8585bb8" (UID: "940f07aa-e2a5-4063-a837-2237c8585bb8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:50.529708 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529622    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert" pod "940f07aa-e2a5-4063-a837-2237c8585bb8" (UID: "940f07aa-e2a5-4063-a837-2237c8585bb8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:50.529708 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529623    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp" pod "940f07aa-e2a5-4063-a837-2237c8585bb8" (UID: "940f07aa-e2a5-4063-a837-2237c8585bb8"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:23:50.529782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.529706    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca" pod "940f07aa-e2a5-4063-a837-2237c8585bb8" (UID: "940f07aa-e2a5-4063-a837-2237c8585bb8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:50.531674 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.531643    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p" pod "940f07aa-e2a5-4063-a837-2237c8585bb8" (UID: "940f07aa-e2a5-4063-a837-2237c8585bb8"). InnerVolumeSpecName "kube-api-access-65p2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:23:50.531674 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.531655    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert" pod "940f07aa-e2a5-4063-a837-2237c8585bb8" (UID: "940f07aa-e2a5-4063-a837-2237c8585bb8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:50.531815 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.531774    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config" pod "940f07aa-e2a5-4063-a837-2237c8585bb8" (UID: "940f07aa-e2a5-4063-a837-2237c8585bb8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:50.630425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.630327    5162 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-console-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:23:50.630425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.630356    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65p2p\" (UniqueName: \"kubernetes.io/projected/940f07aa-e2a5-4063-a837-2237c8585bb8-kube-api-access-65p2p\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:23:50.630425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.630369    5162 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:23:50.630425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.630383    5162 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/940f07aa-e2a5-4063-a837-2237c8585bb8-tmp\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:23:50.630425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.630410    5162 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-serving-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:23:50.630425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.630423    5162 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/940f07aa-e2a5-4063-a837-2237c8585bb8-console-oauth-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:23:50.630733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:50.630440    5162 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/940f07aa-e2a5-4063-a837-2237c8585bb8-oauth-serving-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:23:51.068191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.068150    5162 generic.go:356] "Generic (PLEG): container finished" podID="940f07aa-e2a5-4063-a837-2237c8585bb8" containerID="f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2" exitCode=0
Nov 03 22:23:51.068382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.068212    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b495744d6-2wdfb" event={"ID":"940f07aa-e2a5-4063-a837-2237c8585bb8","Type":"ContainerDied","Data":"f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2"}
Nov 03 22:23:51.068382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.068215    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b495744d6-2wdfb"
Nov 03 22:23:51.068382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.068254    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b495744d6-2wdfb" event={"ID":"940f07aa-e2a5-4063-a837-2237c8585bb8","Type":"ContainerDied","Data":"954a5eaa5f345fa38a01f7bf05103f99ec97f1c4a070fff44371839edeba8156"}
Nov 03 22:23:51.068382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.068273    5162 scope.go:122] "RemoveContainer" containerID="f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2"
Nov 03 22:23:51.076194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.076174    5162 scope.go:122] "RemoveContainer" containerID="f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2"
Nov 03 22:23:51.076463 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:23:51.076444    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2\": container with ID starting with f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2 not found: ID does not exist" containerID="f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2"
Nov 03 22:23:51.076516 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.076472    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2"} err="failed to get container status \"f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2\": rpc error: code = NotFound desc = could not find container \"f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2\": container with ID starting with f14c5ea5b6ebfa0615ad0aed3b22ffc93aa6b582a49baefa2c0a35ebfbb86ca2 not found: ID does not exist"
Nov 03 22:23:51.236735 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.236704    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b495744d6-2wdfb"]
Nov 03 22:23:51.288216 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:51.288186    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b495744d6-2wdfb"]
Nov 03 22:23:52.851271 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:52.851243    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="940f07aa-e2a5-4063-a837-2237c8585bb8" path="/var/lib/kubelet/pods/940f07aa-e2a5-4063-a837-2237c8585bb8/volumes"
Nov 03 22:23:59.210459 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:59.210418    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:23:59.408825 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:23:59.408790    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:00.083235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:00.083205    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:00.290045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:00.290012    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:01.023961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.023929    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-84cc8586df-6lk6j"]
Nov 03 22:24:01.024225 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.024213    5162 cpu_manager.go:414] "containerMap: removing container" podUID="940f07aa-e2a5-4063-a837-2237c8585bb8" containerName="console"
Nov 03 22:24:01.024269 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.024226    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="940f07aa-e2a5-4063-a837-2237c8585bb8" containerName="console"
Nov 03 22:24:01.024302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.024286    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="940f07aa-e2a5-4063-a837-2237c8585bb8" containerName="console"
Nov 03 22:24:01.029100 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.029084    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124424 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124378    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-serving-cert\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124573 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124428    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-trusted-ca-bundle\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124573 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124538    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-oauth-serving-cert\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124579    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-service-ca\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124611    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-config\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124707 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124649    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-oauth-config\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124707 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124674    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k6jg\" (UniqueName: \"kubernetes.io/projected/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-kube-api-access-5k6jg\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.124767 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.124747    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-tmp\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.138224 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.138206    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cc8586df-6lk6j"]
Nov 03 22:24:01.225680 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225652    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-oauth-serving-cert\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.225829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225691    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-service-ca\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.225829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225712    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-config\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.225829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225755    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-oauth-config\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.225829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225800    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k6jg\" (UniqueName: \"kubernetes.io/projected/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-kube-api-access-5k6jg\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.225829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225821    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-tmp\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.226098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225846    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-serving-cert\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.226098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.225863    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-trusted-ca-bundle\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.226274 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.226221    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-tmp\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.226465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.226448    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-config\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.226546 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.226522    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-oauth-serving-cert\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.226750 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.226733    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-service-ca\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.226830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.226809    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-trusted-ca-bundle\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.228137 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.228116    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-oauth-config\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.228287 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.228268    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-console-serving-cert\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.288615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.288545    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k6jg\" (UniqueName: \"kubernetes.io/projected/9a9c3e59-701f-4005-b9cb-3927aefaa9c4-kube-api-access-5k6jg\") pod \"console-84cc8586df-6lk6j\" (UID: \"9a9c3e59-701f-4005-b9cb-3927aefaa9c4\") " pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.339752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.339722    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:01.365662 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.365631    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:01.613773 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.613696    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:01.708746 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:24:01.708717    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9c3e59_701f_4005_b9cb_3927aefaa9c4.slice/crio-e7b404541565aeb88fd6eb6071cc3b9301cc19925b5e43e2ddbb2babe452198b WatchSource:0}: Error finding container e7b404541565aeb88fd6eb6071cc3b9301cc19925b5e43e2ddbb2babe452198b: Status 404 returned error can't find the container with id e7b404541565aeb88fd6eb6071cc3b9301cc19925b5e43e2ddbb2babe452198b
Nov 03 22:24:01.710121 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.710096    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cc8586df-6lk6j"]
Nov 03 22:24:01.815758 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:01.815724    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:02.107009 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:02.106968    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cc8586df-6lk6j" event={"ID":"9a9c3e59-701f-4005-b9cb-3927aefaa9c4","Type":"ContainerStarted","Data":"7771bb83fc4a14f2bba5fcaccaf7fd022eb1203cce1be2ca30a11278ad7e000a"}
Nov 03 22:24:02.107199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:02.107023    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cc8586df-6lk6j" event={"ID":"9a9c3e59-701f-4005-b9cb-3927aefaa9c4","Type":"ContainerStarted","Data":"e7b404541565aeb88fd6eb6071cc3b9301cc19925b5e43e2ddbb2babe452198b"}
Nov 03 22:24:02.228482 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:02.228422    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-84cc8586df-6lk6j" podStartSLOduration=2.228405369 podStartE2EDuration="2.228405369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:24:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:24:02.227735862 +0000 UTC m=+45.492435874" watchObservedRunningTime="2026-11-03 22:24:02.228405369 +0000 UTC m=+45.493105372"
Nov 03 22:24:03.189987 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.189952    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-655dc496c7-lzcqw" podUID="b490753a-2874-4ba6-9fb3-fa9fce802e9e" containerName="console" containerID="cri-o://d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1" gracePeriod=15
Nov 03 22:24:03.307446 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.307420    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:03.439530 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.439507    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:24:03.533576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.533496    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:03.545432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545411    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-oauth-serving-cert\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.545504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545462    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-trusted-ca-bundle\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.545504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545483    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-service-ca\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-service-ca\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.545504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545498    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-config\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-config\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.545640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545534    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b490753a-2874-4ba6-9fb3-fa9fce802e9e-kube-api-access-c968b\" (UniqueName: \"kubernetes.io/projected/b490753a-2874-4ba6-9fb3-fa9fce802e9e-kube-api-access-c968b\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.545640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545568    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-serving-cert\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.545640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545597    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-oauth-config\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.545781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.545636    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b490753a-2874-4ba6-9fb3-fa9fce802e9e-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490753a-2874-4ba6-9fb3-fa9fce802e9e-tmp\") pod \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\" (UID: \"b490753a-2874-4ba6-9fb3-fa9fce802e9e\") "
Nov 03 22:24:03.546061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.546035    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-service-ca" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:03.546192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.546055    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b490753a-2874-4ba6-9fb3-fa9fce802e9e-tmp" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:24:03.546192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.546085    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-config" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:03.546192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.546157    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-oauth-serving-cert" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:03.546334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.546226    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-trusted-ca-bundle" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:03.547776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.547756    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-oauth-config" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:03.547867 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.547803    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b490753a-2874-4ba6-9fb3-fa9fce802e9e-kube-api-access-c968b" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "kube-api-access-c968b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:24:03.547867 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.547808    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-serving-cert" pod "b490753a-2874-4ba6-9fb3-fa9fce802e9e" (UID: "b490753a-2874-4ba6-9fb3-fa9fce802e9e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:03.646655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646618    5162 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-oauth-serving-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:03.646655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646646    5162 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-trusted-ca-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:03.646655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646657    5162 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:03.646883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646666    5162 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:03.646883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646676    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c968b\" (UniqueName: \"kubernetes.io/projected/b490753a-2874-4ba6-9fb3-fa9fce802e9e-kube-api-access-c968b\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:03.646883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646685    5162 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-serving-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:03.646883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646693    5162 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b490753a-2874-4ba6-9fb3-fa9fce802e9e-console-oauth-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:03.646883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:03.646700    5162 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b490753a-2874-4ba6-9fb3-fa9fce802e9e-tmp\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:04.116149 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.116107    5162 generic.go:356] "Generic (PLEG): container finished" podID="b490753a-2874-4ba6-9fb3-fa9fce802e9e" containerID="d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1" exitCode=0
Nov 03 22:24:04.116340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.116151    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-655dc496c7-lzcqw" event={"ID":"b490753a-2874-4ba6-9fb3-fa9fce802e9e","Type":"ContainerDied","Data":"d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1"}
Nov 03 22:24:04.116340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.116200    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-655dc496c7-lzcqw" event={"ID":"b490753a-2874-4ba6-9fb3-fa9fce802e9e","Type":"ContainerDied","Data":"575173b253283ec57d89d80ded4f78533b6def5b3bec4cd471243dc567a59603"}
Nov 03 22:24:04.116340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.116221    5162 scope.go:122] "RemoveContainer" containerID="d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1"
Nov 03 22:24:04.116340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.116158    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-655dc496c7-lzcqw"
Nov 03 22:24:04.124588 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.124568    5162 scope.go:122] "RemoveContainer" containerID="d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1"
Nov 03 22:24:04.124835 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:04.124819    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1\": container with ID starting with d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1 not found: ID does not exist" containerID="d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1"
Nov 03 22:24:04.124902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.124842    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1"} err="failed to get container status \"d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1\": rpc error: code = NotFound desc = could not find container \"d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1\": container with ID starting with d087d95f6c62dafbc264ae2cdaaf7b17825fe6b94e281b40e41b3c9b2eb1ebe1 not found: ID does not exist"
Nov 03 22:24:04.285375 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.285345    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-655dc496c7-lzcqw"]
Nov 03 22:24:04.337151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.337124    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-655dc496c7-lzcqw"]
Nov 03 22:24:04.851409 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:04.851363    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b490753a-2874-4ba6-9fb3-fa9fce802e9e" path="/var/lib/kubelet/pods/b490753a-2874-4ba6-9fb3-fa9fce802e9e/volumes"
Nov 03 22:24:10.082500 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:10.082461    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:24:11.340624 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:11.340594    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:11.341063 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:11.340650    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:11.345294 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:11.345273    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:12.149556 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.149525    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84cc8586df-6lk6j"
Nov 03 22:24:12.324030 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.324001    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94b457789-fvcgc"]
Nov 03 22:24:12.563094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.563065    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:24:12.563529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.563370    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="prometheus" containerID="cri-o://0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470" gracePeriod=600
Nov 03 22:24:12.563529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.563419    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6" gracePeriod=600
Nov 03 22:24:12.563529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.563494    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="thanos-sidecar" containerID="cri-o://2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402" gracePeriod=600
Nov 03 22:24:12.563706 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.563473    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-web" containerID="cri-o://a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f" gracePeriod=600
Nov 03 22:24:12.563706 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.563548    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="config-reloader" containerID="cri-o://c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded" gracePeriod=600
Nov 03 22:24:12.563706 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.563374    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy" containerID="cri-o://c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b" gracePeriod=600
Nov 03 22:24:12.828569 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.828543    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:12.924020 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.923995    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-web-config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-web-config\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924030    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-tls-assets\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-tls-assets\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924053    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-kube-api-access-w9q7f\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-kube-api-access-w9q7f\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924071    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-metrics-client-ca\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924093    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-metrics-client-certs\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924116    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-config-out\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924135    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-2\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924164    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-grpc-tls\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924186    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-serving-certs-ca-bundle\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924213    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924248    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-0\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924276    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-kubelet-serving-ca-bundle\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924325    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-tls\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924354    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924383    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-thanos-prometheus-http-client-file\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924457    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-config\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924485    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-trusted-ca-bundle\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924498    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-metrics-client-ca" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:12.924559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924526    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-1\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.925106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924842    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-1" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:12.925106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924886    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-2" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:12.925106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924933    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-kubelet-serving-ca-bundle" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:12.925106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.924983    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-db\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.925106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.925031    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-kube-rbac-proxy\") pod \"9e40242d-93d1-4ee5-8028-f180905714d8\" (UID: \"9e40242d-93d1-4ee5-8028-f180905714d8\") "
Nov 03 22:24:12.925372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.925266    5162 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-metrics-client-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:12.925372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.925285    5162 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:12.925372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.925302    5162 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:12.925372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.925316    5162 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:12.925644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.925619    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-trusted-ca-bundle" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:12.926876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.926843    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-kube-api-access-w9q7f" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "kube-api-access-w9q7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:24:12.927628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.927603    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-serving-certs-ca-bundle" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:12.928381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928347    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-config" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.928381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928372    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-kube-rbac-proxy" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.928551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928371    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-db" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:24:12.928551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928509    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-tls" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.928791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928732    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-0" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:12.928791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928739    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-tls-assets" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:24:12.928791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928742    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-config-out" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:24:12.928791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928763    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-metrics-client-certs" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.928976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.928927    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-kube-rbac-proxy-web" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.929533 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.929514    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-thanos-prometheus-http-client-file" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.929797 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.929779    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-grpc-tls" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.930009 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.929989    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-thanos-sidecar-tls" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:12.937988 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:12.937966    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-web-config" pod "9e40242d-93d1-4ee5-8028-f180905714d8" (UID: "9e40242d-93d1-4ee5-8028-f180905714d8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:13.026646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026612    5162 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-db\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026640    5162 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-kube-rbac-proxy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026669    5162 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-web-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026678    5162 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-tls-assets\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026687    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9q7f\" (UniqueName: \"kubernetes.io/projected/9e40242d-93d1-4ee5-8028-f180905714d8-kube-api-access-w9q7f\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026695    5162 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-metrics-client-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026704    5162 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e40242d-93d1-4ee5-8028-f180905714d8-config-out\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026712    5162 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-grpc-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026721    5162 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026731    5162 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026740    5162 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026749    5162 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026757    5162 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026766    5162 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026774    5162 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e40242d-93d1-4ee5-8028-f180905714d8-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.026823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.026782    5162 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e40242d-93d1-4ee5-8028-f180905714d8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:13.151976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151897    5162 generic.go:356] "Generic (PLEG): container finished" podID="9e40242d-93d1-4ee5-8028-f180905714d8" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6" exitCode=0
Nov 03 22:24:13.151976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151930    5162 generic.go:356] "Generic (PLEG): container finished" podID="9e40242d-93d1-4ee5-8028-f180905714d8" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b" exitCode=0
Nov 03 22:24:13.151976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151939    5162 generic.go:356] "Generic (PLEG): container finished" podID="9e40242d-93d1-4ee5-8028-f180905714d8" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f" exitCode=0
Nov 03 22:24:13.151976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151948    5162 generic.go:356] "Generic (PLEG): container finished" podID="9e40242d-93d1-4ee5-8028-f180905714d8" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402" exitCode=0
Nov 03 22:24:13.151976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151956    5162 generic.go:356] "Generic (PLEG): container finished" podID="9e40242d-93d1-4ee5-8028-f180905714d8" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded" exitCode=0
Nov 03 22:24:13.151976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151964    5162 generic.go:356] "Generic (PLEG): container finished" podID="9e40242d-93d1-4ee5-8028-f180905714d8" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470" exitCode=0
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151984    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"}
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.152050    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"}
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.151997    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.152083    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"}
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.152109    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"}
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.152134    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"}
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.152149    5162 scope.go:122] "RemoveContainer" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.152172    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"}
Nov 03 22:24:13.152247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.152209    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e40242d-93d1-4ee5-8028-f180905714d8","Type":"ContainerDied","Data":"edc2d6af90269ad11493e4dc4531babc108dce8831be45a50097363b43c988a4"}
Nov 03 22:24:13.159678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.159660    5162 scope.go:122] "RemoveContainer" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.166271 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.166256    5162 scope.go:122] "RemoveContainer" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.172571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.172555    5162 scope.go:122] "RemoveContainer" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.178559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.178543    5162 scope.go:122] "RemoveContainer" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.184841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.184825    5162 scope.go:122] "RemoveContainer" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.191362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.191347    5162 scope.go:122] "RemoveContainer" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.197268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.197253    5162 scope.go:122] "RemoveContainer" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.197553 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:13.197535    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": container with ID starting with 3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6 not found: ID does not exist" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.197621 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.197560    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"} err="failed to get container status \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": rpc error: code = NotFound desc = could not find container \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": container with ID starting with 3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6 not found: ID does not exist"
Nov 03 22:24:13.197621 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.197578    5162 scope.go:122] "RemoveContainer" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.197830 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:13.197815    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": container with ID starting with c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b not found: ID does not exist" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.197875 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.197833    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"} err="failed to get container status \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": rpc error: code = NotFound desc = could not find container \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": container with ID starting with c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b not found: ID does not exist"
Nov 03 22:24:13.197875 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.197847    5162 scope.go:122] "RemoveContainer" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.198095 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:13.198075    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": container with ID starting with a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f not found: ID does not exist" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.198141 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.198100    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"} err="failed to get container status \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": rpc error: code = NotFound desc = could not find container \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": container with ID starting with a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f not found: ID does not exist"
Nov 03 22:24:13.198141 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.198117    5162 scope.go:122] "RemoveContainer" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.198380 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:13.198361    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": container with ID starting with 2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402 not found: ID does not exist" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.198478 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.198385    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"} err="failed to get container status \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": rpc error: code = NotFound desc = could not find container \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": container with ID starting with 2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402 not found: ID does not exist"
Nov 03 22:24:13.198522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.198483    5162 scope.go:122] "RemoveContainer" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.198745 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:13.198728    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": container with ID starting with c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded not found: ID does not exist" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.198782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.198750    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"} err="failed to get container status \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": rpc error: code = NotFound desc = could not find container \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": container with ID starting with c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded not found: ID does not exist"
Nov 03 22:24:13.198782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.198766    5162 scope.go:122] "RemoveContainer" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.199006 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:13.198985    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": container with ID starting with 0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470 not found: ID does not exist" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.199054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199005    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"} err="failed to get container status \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": rpc error: code = NotFound desc = could not find container \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": container with ID starting with 0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470 not found: ID does not exist"
Nov 03 22:24:13.199054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199020    5162 scope.go:122] "RemoveContainer" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.199257 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:13.199241    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": container with ID starting with 2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124 not found: ID does not exist" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.199293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199262    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"} err="failed to get container status \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": rpc error: code = NotFound desc = could not find container \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": container with ID starting with 2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124 not found: ID does not exist"
Nov 03 22:24:13.199293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199277    5162 scope.go:122] "RemoveContainer" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.199522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199505    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"} err="failed to get container status \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": rpc error: code = NotFound desc = could not find container \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": container with ID starting with 3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6 not found: ID does not exist"
Nov 03 22:24:13.199522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199522    5162 scope.go:122] "RemoveContainer" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.199743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199726    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"} err="failed to get container status \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": rpc error: code = NotFound desc = could not find container \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": container with ID starting with c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b not found: ID does not exist"
Nov 03 22:24:13.199743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199743    5162 scope.go:122] "RemoveContainer" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.199957 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199942    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"} err="failed to get container status \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": rpc error: code = NotFound desc = could not find container \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": container with ID starting with a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f not found: ID does not exist"
Nov 03 22:24:13.199998 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.199957    5162 scope.go:122] "RemoveContainer" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.200187 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200170    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"} err="failed to get container status \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": rpc error: code = NotFound desc = could not find container \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": container with ID starting with 2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402 not found: ID does not exist"
Nov 03 22:24:13.200234 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200187    5162 scope.go:122] "RemoveContainer" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.200419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200400    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"} err="failed to get container status \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": rpc error: code = NotFound desc = could not find container \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": container with ID starting with c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded not found: ID does not exist"
Nov 03 22:24:13.200469 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200420    5162 scope.go:122] "RemoveContainer" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.200632 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200618    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"} err="failed to get container status \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": rpc error: code = NotFound desc = could not find container \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": container with ID starting with 0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470 not found: ID does not exist"
Nov 03 22:24:13.200672 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200634    5162 scope.go:122] "RemoveContainer" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.200834 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200819    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"} err="failed to get container status \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": rpc error: code = NotFound desc = could not find container \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": container with ID starting with 2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124 not found: ID does not exist"
Nov 03 22:24:13.200875 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.200834    5162 scope.go:122] "RemoveContainer" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.201045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201028    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"} err="failed to get container status \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": rpc error: code = NotFound desc = could not find container \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": container with ID starting with 3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6 not found: ID does not exist"
Nov 03 22:24:13.201093 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201046    5162 scope.go:122] "RemoveContainer" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.201264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201240    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"} err="failed to get container status \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": rpc error: code = NotFound desc = could not find container \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": container with ID starting with c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b not found: ID does not exist"
Nov 03 22:24:13.201264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201264    5162 scope.go:122] "RemoveContainer" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.201547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201530    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"} err="failed to get container status \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": rpc error: code = NotFound desc = could not find container \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": container with ID starting with a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f not found: ID does not exist"
Nov 03 22:24:13.201598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201557    5162 scope.go:122] "RemoveContainer" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.201770 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201754    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"} err="failed to get container status \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": rpc error: code = NotFound desc = could not find container \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": container with ID starting with 2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402 not found: ID does not exist"
Nov 03 22:24:13.201811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201771    5162 scope.go:122] "RemoveContainer" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.201995 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201976    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"} err="failed to get container status \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": rpc error: code = NotFound desc = could not find container \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": container with ID starting with c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded not found: ID does not exist"
Nov 03 22:24:13.201995 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.201994    5162 scope.go:122] "RemoveContainer" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.202212 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202197    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"} err="failed to get container status \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": rpc error: code = NotFound desc = could not find container \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": container with ID starting with 0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470 not found: ID does not exist"
Nov 03 22:24:13.202259 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202212    5162 scope.go:122] "RemoveContainer" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.202440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202425    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"} err="failed to get container status \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": rpc error: code = NotFound desc = could not find container \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": container with ID starting with 2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124 not found: ID does not exist"
Nov 03 22:24:13.202489 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202440    5162 scope.go:122] "RemoveContainer" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.202643 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202629    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"} err="failed to get container status \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": rpc error: code = NotFound desc = could not find container \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": container with ID starting with 3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6 not found: ID does not exist"
Nov 03 22:24:13.202691 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202643    5162 scope.go:122] "RemoveContainer" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.202842 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202826    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"} err="failed to get container status \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": rpc error: code = NotFound desc = could not find container \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": container with ID starting with c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b not found: ID does not exist"
Nov 03 22:24:13.202883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.202844    5162 scope.go:122] "RemoveContainer" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.203039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203022    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"} err="failed to get container status \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": rpc error: code = NotFound desc = could not find container \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": container with ID starting with a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f not found: ID does not exist"
Nov 03 22:24:13.203082 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203040    5162 scope.go:122] "RemoveContainer" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.203235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203220    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"} err="failed to get container status \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": rpc error: code = NotFound desc = could not find container \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": container with ID starting with 2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402 not found: ID does not exist"
Nov 03 22:24:13.203275 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203235    5162 scope.go:122] "RemoveContainer" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.203466 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203449    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"} err="failed to get container status \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": rpc error: code = NotFound desc = could not find container \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": container with ID starting with c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded not found: ID does not exist"
Nov 03 22:24:13.203511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203467    5162 scope.go:122] "RemoveContainer" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.203723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203706    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"} err="failed to get container status \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": rpc error: code = NotFound desc = could not find container \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": container with ID starting with 0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470 not found: ID does not exist"
Nov 03 22:24:13.203764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203723    5162 scope.go:122] "RemoveContainer" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.203915 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203899    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"} err="failed to get container status \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": rpc error: code = NotFound desc = could not find container \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": container with ID starting with 2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124 not found: ID does not exist"
Nov 03 22:24:13.203951 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.203915    5162 scope.go:122] "RemoveContainer" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.204137 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204122    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"} err="failed to get container status \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": rpc error: code = NotFound desc = could not find container \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": container with ID starting with 3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6 not found: ID does not exist"
Nov 03 22:24:13.204137 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204136    5162 scope.go:122] "RemoveContainer" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.204352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204335    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"} err="failed to get container status \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": rpc error: code = NotFound desc = could not find container \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": container with ID starting with c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b not found: ID does not exist"
Nov 03 22:24:13.204434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204352    5162 scope.go:122] "RemoveContainer" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.204593 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204575    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"} err="failed to get container status \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": rpc error: code = NotFound desc = could not find container \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": container with ID starting with a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f not found: ID does not exist"
Nov 03 22:24:13.204644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204593    5162 scope.go:122] "RemoveContainer" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.204807 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204791    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"} err="failed to get container status \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": rpc error: code = NotFound desc = could not find container \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": container with ID starting with 2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402 not found: ID does not exist"
Nov 03 22:24:13.204850 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.204806    5162 scope.go:122] "RemoveContainer" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.205061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205045    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"} err="failed to get container status \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": rpc error: code = NotFound desc = could not find container \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": container with ID starting with c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded not found: ID does not exist"
Nov 03 22:24:13.205110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205061    5162 scope.go:122] "RemoveContainer" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.205269 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205254    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"} err="failed to get container status \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": rpc error: code = NotFound desc = could not find container \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": container with ID starting with 0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470 not found: ID does not exist"
Nov 03 22:24:13.205307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205270    5162 scope.go:122] "RemoveContainer" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.205517 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205500    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"} err="failed to get container status \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": rpc error: code = NotFound desc = could not find container \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": container with ID starting with 2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124 not found: ID does not exist"
Nov 03 22:24:13.205517 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205517    5162 scope.go:122] "RemoveContainer" containerID="3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"
Nov 03 22:24:13.205761 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205744    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6"} err="failed to get container status \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": rpc error: code = NotFound desc = could not find container \"3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6\": container with ID starting with 3484c58d03b6664fa4813754c5b346273d8411bb218295eac797e260295337a6 not found: ID does not exist"
Nov 03 22:24:13.205857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205762    5162 scope.go:122] "RemoveContainer" containerID="c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"
Nov 03 22:24:13.205979 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205960    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b"} err="failed to get container status \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": rpc error: code = NotFound desc = could not find container \"c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b\": container with ID starting with c8f3966432abb0c5b046d6c9210c3607e62f1b1d330ca9d85918a9eed79f313b not found: ID does not exist"
Nov 03 22:24:13.206020 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.205979    5162 scope.go:122] "RemoveContainer" containerID="a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"
Nov 03 22:24:13.206203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206175    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f"} err="failed to get container status \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": rpc error: code = NotFound desc = could not find container \"a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f\": container with ID starting with a3983dcce5c47d99715281f454b92e50e2eab1b3d2e039661c5c2f49a2f6341f not found: ID does not exist"
Nov 03 22:24:13.206261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206202    5162 scope.go:122] "RemoveContainer" containerID="2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"
Nov 03 22:24:13.206444 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206425    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402"} err="failed to get container status \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": rpc error: code = NotFound desc = could not find container \"2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402\": container with ID starting with 2c251bf085c23a568bbcbdd97b7ab5893a8d68ee707a529bc378b73d9892b402 not found: ID does not exist"
Nov 03 22:24:13.206513 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206445    5162 scope.go:122] "RemoveContainer" containerID="c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"
Nov 03 22:24:13.206684 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206657    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded"} err="failed to get container status \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": rpc error: code = NotFound desc = could not find container \"c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded\": container with ID starting with c72beacf462c0f769139ebd1ab99ee8d6e80048c2abb4abe1140f74e87c86ded not found: ID does not exist"
Nov 03 22:24:13.206684 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206681    5162 scope.go:122] "RemoveContainer" containerID="0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"
Nov 03 22:24:13.206971 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206954    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470"} err="failed to get container status \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": rpc error: code = NotFound desc = could not find container \"0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470\": container with ID starting with 0c3b0494375ebe97f5ea4c567d76757d9ea52099ed34d62e699dcda68a94b470 not found: ID does not exist"
Nov 03 22:24:13.206971 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.206970    5162 scope.go:122] "RemoveContainer" containerID="2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"
Nov 03 22:24:13.207239 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.207218    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124"} err="failed to get container status \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": rpc error: code = NotFound desc = could not find container \"2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124\": container with ID starting with 2c24fc6200a2310c59d0fc66dee7e51f4b8935002161d7a9f9ef5cc2ce7ae124 not found: ID does not exist"
Nov 03 22:24:13.324764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.324731    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:24:13.329172 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.329154    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:24:13.368703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368676    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:24:13.368947 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368934    5162 cpu_manager.go:414] "containerMap: removing container" podUID="b490753a-2874-4ba6-9fb3-fa9fce802e9e" containerName="console"
Nov 03 22:24:13.368985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368948    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b490753a-2874-4ba6-9fb3-fa9fce802e9e" containerName="console"
Nov 03 22:24:13.368985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368957    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy"
Nov 03 22:24:13.368985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368962    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy"
Nov 03 22:24:13.368985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368971    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="init-config-reloader"
Nov 03 22:24:13.368985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368976    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="init-config-reloader"
Nov 03 22:24:13.368985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368986    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="prometheus"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368991    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="prometheus"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.368998    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-thanos"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369002    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-thanos"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369012    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="config-reloader"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369017    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="config-reloader"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369031    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="thanos-sidecar"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369035    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="thanos-sidecar"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369046    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-web"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369050    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-web"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369112    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="thanos-sidecar"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369120    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-web"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369125    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="b490753a-2874-4ba6-9fb3-fa9fce802e9e" containerName="console"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369132    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369138    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="prometheus"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369143    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="kube-rbac-proxy-thanos"
Nov 03 22:24:13.369161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.369149    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" containerName="config-reloader"
Nov 03 22:24:13.374381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.374361    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.429568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429541    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 03 22:24:13.429757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429679    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6f987d-e21b-4888-b9a6-6320d52d7b85-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.429757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429704    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.429757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429724    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.429757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429740    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429763    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429825    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429829    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429849    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429866    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429892    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429902    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429951    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.429985    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430043    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430042    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.430077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430074    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-config\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430096    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430121    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8nxw\" (UniqueName: \"kubernetes.io/projected/8b6f987d-e21b-4888-b9a6-6320d52d7b85-kube-api-access-x8nxw\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430151    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430153    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430157    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42qr9\""
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430202    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430202    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430219    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430245    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6f987d-e21b-4888-b9a6-6320d52d7b85-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430301    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430333    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430405    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430432    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7h3uqh5pcgour\""
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430456    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430507    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430553    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.431130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.430587    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.482045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.482024    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 03 22:24:13.482045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.482030    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 03 22:24:13.497792 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.497766    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:24:13.531799 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.531772    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.531901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.531806    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.531901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.531832    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.531972 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.531955    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6f987d-e21b-4888-b9a6-6320d52d7b85-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532016 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.531986    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532016 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532006    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532115 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532022    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532115 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532052    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532171    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532206    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532326 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532240    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532326 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532273    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-config\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532326 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532296    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532515 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532323    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8nxw\" (UniqueName: \"kubernetes.io/projected/8b6f987d-e21b-4888-b9a6-6320d52d7b85-kube-api-access-x8nxw\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532515 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532356    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532515 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532381    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.532515 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.532430    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6f987d-e21b-4888-b9a6-6320d52d7b85-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.534616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.533984    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.534616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.533986    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.534616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.534441    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.534616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.534484    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.534616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.534527    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.534979    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.535032    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.535413    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.535422    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.535471    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.535670    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.535764    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.535878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.535834    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.536373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.536063    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.536689 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.536646    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b6f987d-e21b-4888-b9a6-6320d52d7b85-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.536783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.536705    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b6f987d-e21b-4888-b9a6-6320d52d7b85-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.536954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.536935    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b6f987d-e21b-4888-b9a6-6320d52d7b85-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.537074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.537059    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.537074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.537063    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.537598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.537577    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.537895 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.537880    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-config\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.538368 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.538353    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b6f987d-e21b-4888-b9a6-6320d52d7b85-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.596294 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.596268    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8nxw\" (UniqueName: \"kubernetes.io/projected/8b6f987d-e21b-4888-b9a6-6320d52d7b85-kube-api-access-x8nxw\") pod \"prometheus-k8s-0\" (UID: \"8b6f987d-e21b-4888-b9a6-6320d52d7b85\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:13.698810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:13.698737    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:14.128436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:14.128410    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 03 22:24:14.157433 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:14.157410    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerStarted","Data":"8aa429709e38a3c52b1bfa9d4312fa8c250f938d386f5e22807dbeb1f26f7f6c"}
Nov 03 22:24:14.851798 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:14.851764    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e40242d-93d1-4ee5-8028-f180905714d8" path="/var/lib/kubelet/pods/9e40242d-93d1-4ee5-8028-f180905714d8/volumes"
Nov 03 22:24:15.162747 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:15.162670    5162 generic.go:356] "Generic (PLEG): container finished" podID="8b6f987d-e21b-4888-b9a6-6320d52d7b85" containerID="4a7a778336ea41b2920fd64b284d13ede9a12df0b9cc800f6166b2da5e0ff882" exitCode=0
Nov 03 22:24:15.162747 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:15.162735    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerDied","Data":"4a7a778336ea41b2920fd64b284d13ede9a12df0b9cc800f6166b2da5e0ff882"}
Nov 03 22:24:16.169138 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:16.169098    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerStarted","Data":"f04a4c86013fd9f4ec453f710ca7b4956340c7855146d84457c99cce28b63f32"}
Nov 03 22:24:16.169538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:16.169181    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerStarted","Data":"b9ac121d2d6a967a39c26a3722b323fa817a28ee2f1b098ad288b9582ea03555"}
Nov 03 22:24:16.169538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:16.169221    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerStarted","Data":"001327eccb5e62947471680c8ad832c43d184ce2505eda58d89f54716871520b"}
Nov 03 22:24:16.169538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:16.169261    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerStarted","Data":"23f78eab3d6f2ca8b322aa2e79f4557679e44544c5b6881b4061a5a2b995270e"}
Nov 03 22:24:16.169538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:16.169302    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerStarted","Data":"8a8a87de033ad9779429cccead12e84d7f922476583d17336d49cd1336ce276e"}
Nov 03 22:24:16.169538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:16.169337    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b6f987d-e21b-4888-b9a6-6320d52d7b85","Type":"ContainerStarted","Data":"7a57823c313076002c4e9bacad1ff1ce9c586b3190f799c21be622442f754c7f"}
Nov 03 22:24:16.305363 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:16.305311    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.2888302830000002 podStartE2EDuration="3.305298514s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.468231ms" isStatefulPod=false podCreationTimestamp="2026-11-03 22:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:24:16.304005709 +0000 UTC m=+59.568705730" watchObservedRunningTime="2026-11-03 22:24:16.305298514 +0000 UTC m=+59.569998573"
Nov 03 22:24:18.699009 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:18.698975    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:22.510337 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:22.510301    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:22.525564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:22.525542    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:23.208060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:23.208034    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 03 22:24:28.105062 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:28.105014    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:32.758869 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:32.758834    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:37.345463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.345422    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-94b457789-fvcgc" podUID="6296a6f9-642b-44c3-8715-1b925c6bf1bf" containerName="console" containerID="cri-o://cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24" gracePeriod=15
Nov 03 22:24:37.591778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.591755    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:24:37.647445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647362    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-service-ca\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-service-ca\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.647445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647435    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-config\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-config\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.647617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647582    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-oauth-config\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.647687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647621    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-serving-cert\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.647687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647640    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-oauth-serving-cert\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.647687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647668    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6296a6f9-642b-44c3-8715-1b925c6bf1bf-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6296a6f9-642b-44c3-8715-1b925c6bf1bf-tmp\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.647833 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647698    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6296a6f9-642b-44c3-8715-1b925c6bf1bf-kube-api-access-dmn8p\" (UniqueName: \"kubernetes.io/projected/6296a6f9-642b-44c3-8715-1b925c6bf1bf-kube-api-access-dmn8p\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.647833 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647715    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-service-ca" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:37.647833 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647767    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-config" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:37.647833 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.647790    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-trusted-ca-bundle\") pod \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\" (UID: \"6296a6f9-642b-44c3-8715-1b925c6bf1bf\") "
Nov 03 22:24:37.648111 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.648090    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-oauth-serving-cert" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:37.648169 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.648144    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6296a6f9-642b-44c3-8715-1b925c6bf1bf-tmp" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:24:37.648264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.648248    5162 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:37.648324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.648266    5162 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:37.648324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.648275    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-trusted-ca-bundle" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:37.648324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.648281    5162 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-oauth-serving-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:37.648324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.648318    5162 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6296a6f9-642b-44c3-8715-1b925c6bf1bf-tmp\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:37.649826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.649808    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-oauth-config" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:37.650184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.650168    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6296a6f9-642b-44c3-8715-1b925c6bf1bf-kube-api-access-dmn8p" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "kube-api-access-dmn8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:24:37.650184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.650169    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-serving-cert" pod "6296a6f9-642b-44c3-8715-1b925c6bf1bf" (UID: "6296a6f9-642b-44c3-8715-1b925c6bf1bf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:37.749073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.749050    5162 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-oauth-config\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:37.749073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.749071    5162 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6296a6f9-642b-44c3-8715-1b925c6bf1bf-console-serving-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:37.749234 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.749080    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmn8p\" (UniqueName: \"kubernetes.io/projected/6296a6f9-642b-44c3-8715-1b925c6bf1bf-kube-api-access-dmn8p\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:37.749234 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:37.749090    5162 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6296a6f9-642b-44c3-8715-1b925c6bf1bf-trusted-ca-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:24:38.244521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.244488    5162 generic.go:356] "Generic (PLEG): container finished" podID="6296a6f9-642b-44c3-8715-1b925c6bf1bf" containerID="cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24" exitCode=0
Nov 03 22:24:38.244521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.244526    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94b457789-fvcgc" event={"ID":"6296a6f9-642b-44c3-8715-1b925c6bf1bf","Type":"ContainerDied","Data":"cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24"}
Nov 03 22:24:38.244741 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.244533    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94b457789-fvcgc"
Nov 03 22:24:38.244741 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.244565    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94b457789-fvcgc" event={"ID":"6296a6f9-642b-44c3-8715-1b925c6bf1bf","Type":"ContainerDied","Data":"4419ca58dc1a8afa9ee0f876512cc4cfcd5d529656c47ea334386c91ddc60da3"}
Nov 03 22:24:38.244741 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.244587    5162 scope.go:122] "RemoveContainer" containerID="cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24"
Nov 03 22:24:38.253044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.253028    5162 scope.go:122] "RemoveContainer" containerID="cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24"
Nov 03 22:24:38.253281 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:38.253266    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24\": container with ID starting with cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24 not found: ID does not exist" containerID="cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24"
Nov 03 22:24:38.253327 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.253289    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24"} err="failed to get container status \"cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24\": rpc error: code = NotFound desc = could not find container \"cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24\": container with ID starting with cb2299845444e9f4574d9ac863c8ef01a09d747a6aac260dea8943beeaea0c24 not found: ID does not exist"
Nov 03 22:24:38.414449 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.414421    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94b457789-fvcgc"]
Nov 03 22:24:38.465933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.465907    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-94b457789-fvcgc"]
Nov 03 22:24:38.851419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:38.851373    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6296a6f9-642b-44c3-8715-1b925c6bf1bf" path="/var/lib/kubelet/pods/6296a6f9-642b-44c3-8715-1b925c6bf1bf/volumes"
Nov 03 22:24:40.082717 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:24:40.082683    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:24:40.893726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:40.893683    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:50.132498 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:50.132458    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:24:57.865159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:24:57.865128    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:01.737955 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:01.737918    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:01.944315 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:01.944279    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:10.082275 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:25:10.082189    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:25:28.416965 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:28.416925    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:33.079352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:33.079312    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:40.081970 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:25:40.081932    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:25:48.608094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:48.608045    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:48.860466 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:48.860375    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:50.231888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:50.231859    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:50.480634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:50.480599    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:51.397468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:51.397432    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:51.597603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:51.597571    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:52.334831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.334793    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"]
Nov 03 22:25:52.335153 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.335137    5162 cpu_manager.go:414] "containerMap: removing container" podUID="6296a6f9-642b-44c3-8715-1b925c6bf1bf" containerName="console"
Nov 03 22:25:52.335235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.335156    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6296a6f9-642b-44c3-8715-1b925c6bf1bf" containerName="console"
Nov 03 22:25:52.335291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.335278    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="6296a6f9-642b-44c3-8715-1b925c6bf1bf" containerName="console"
Nov 03 22:25:52.338276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.338255    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.392984 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.392965    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 22:25:52.427303 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.427276    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.427654 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.427311    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk5dh\" (UniqueName: \"kubernetes.io/projected/5d3da421-41cc-4400-a8ab-2a7a14c91462-kube-api-access-mk5dh\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.427654 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.427340    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.444566 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.444540    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 22:25:52.444664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.444640    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-5c4bg\""
Nov 03 22:25:52.445626 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.445611    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"]
Nov 03 22:25:52.528280 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.528252    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.528411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.528293    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk5dh\" (UniqueName: \"kubernetes.io/projected/5d3da421-41cc-4400-a8ab-2a7a14c91462-kube-api-access-mk5dh\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.528411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.528326    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.528610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.528594    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.528649 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.528634    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.591243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.591190    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk5dh\" (UniqueName: \"kubernetes.io/projected/5d3da421-41cc-4400-a8ab-2a7a14c91462-kube-api-access-mk5dh\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:52.653757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:52.653726    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:25:53.019152 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:25:53.019110    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3da421_41cc_4400_a8ab_2a7a14c91462.slice/crio-1f691dcbe890c1c3b21e59eaca57cc1025a00167679b473df929a9ceabf6ff2f WatchSource:0}: Error finding container 1f691dcbe890c1c3b21e59eaca57cc1025a00167679b473df929a9ceabf6ff2f: Status 404 returned error can't find the container with id 1f691dcbe890c1c3b21e59eaca57cc1025a00167679b473df929a9ceabf6ff2f
Nov 03 22:25:53.020200 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:53.020174    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"]
Nov 03 22:25:53.495159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:53.495117    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm" event={"ID":"5d3da421-41cc-4400-a8ab-2a7a14c91462","Type":"ContainerStarted","Data":"1f691dcbe890c1c3b21e59eaca57cc1025a00167679b473df929a9ceabf6ff2f"}
Nov 03 22:25:54.188686 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.188654    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"]
Nov 03 22:25:54.192281 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.192260    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.239124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.239093    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.239263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.239153    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q58x\" (UniqueName: \"kubernetes.io/projected/d159095a-05c8-4eeb-a501-c6ca0935c4a9-kube-api-access-2q58x\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.239263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.239194    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.297889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.297856    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-mdfgz\""
Nov 03 22:25:54.299524 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.299491    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"]
Nov 03 22:25:54.300381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.300360    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"]
Nov 03 22:25:54.304084 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.304065    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.340494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.340467    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q58x\" (UniqueName: \"kubernetes.io/projected/d159095a-05c8-4eeb-a501-c6ca0935c4a9-kube-api-access-2q58x\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.340614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.340509    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.340614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.340546    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.340614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.340568    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.340614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.340597    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzp7b\" (UniqueName: \"kubernetes.io/projected/4cd4c48c-804d-4c71-975d-07954eea0274-kube-api-access-mzp7b\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.340963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.340944    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.341090 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.340946    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.341205 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.341183    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.358357 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.358325    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-55g2t\""
Nov 03 22:25:54.362748 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.362728    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"]
Nov 03 22:25:54.388482 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.388459    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"]
Nov 03 22:25:54.391977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.391959    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.403736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.403710    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q58x\" (UniqueName: \"kubernetes.io/projected/d159095a-05c8-4eeb-a501-c6ca0935c4a9-kube-api-access-2q58x\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.442334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442264    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5nbz\" (UniqueName: \"kubernetes.io/projected/5aa1f333-5c2e-46c6-acfe-09641db947ec-kube-api-access-j5nbz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.442334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442304    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.442600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442344    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.442600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442484    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzp7b\" (UniqueName: \"kubernetes.io/projected/4cd4c48c-804d-4c71-975d-07954eea0274-kube-api-access-mzp7b\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.442600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442545    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.442600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442597    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.442814 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442753    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.442945 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.442920    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.451276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.451252    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"]
Nov 03 22:25:54.458461 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.458438    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-bqw9g\""
Nov 03 22:25:54.517950 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.517697    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:25:54.534885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.525269    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"]
Nov 03 22:25:54.534885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.532627    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.535974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.535947    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzp7b\" (UniqueName: \"kubernetes.io/projected/4cd4c48c-804d-4c71-975d-07954eea0274-kube-api-access-mzp7b\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.543900 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.543871    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.544028 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.543938    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5nbz\" (UniqueName: \"kubernetes.io/projected/5aa1f333-5c2e-46c6-acfe-09641db947ec-kube-api-access-j5nbz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.544028 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.543971    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.544300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.544275    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.544576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.544558    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.587669 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.587573    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"]
Nov 03 22:25:54.587669 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.587633    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-jcsvk\""
Nov 03 22:25:54.607871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.607839    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5nbz\" (UniqueName: \"kubernetes.io/projected/5aa1f333-5c2e-46c6-acfe-09641db947ec-kube-api-access-j5nbz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.621587 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.621559    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:25:54.644980 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.644953    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.645116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.645003    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.645177 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.645119    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t96s\" (UniqueName: \"kubernetes.io/projected/6b9e5c5d-7a5b-4d87-8710-030e9d175426-kube-api-access-6t96s\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.708320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.708250    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:25:54.746442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.746411    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.746600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.746464    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.746600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.746516    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t96s\" (UniqueName: \"kubernetes.io/projected/6b9e5c5d-7a5b-4d87-8710-030e9d175426-kube-api-access-6t96s\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.747073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.747047    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.747159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.747077    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.810109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.810083    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t96s\" (UniqueName: \"kubernetes.io/projected/6b9e5c5d-7a5b-4d87-8710-030e9d175426-kube-api-access-6t96s\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.873354 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.873324    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:25:54.911456 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:25:54.911428    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd159095a_05c8_4eeb_a501_c6ca0935c4a9.slice/crio-c766ac5be2f5e8959a4120f81ac0d5c571c04790388dcc81203edfa15e5a98fe WatchSource:0}: Error finding container c766ac5be2f5e8959a4120f81ac0d5c571c04790388dcc81203edfa15e5a98fe: Status 404 returned error can't find the container with id c766ac5be2f5e8959a4120f81ac0d5c571c04790388dcc81203edfa15e5a98fe
Nov 03 22:25:54.913431 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:54.913406    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"]
Nov 03 22:25:55.008305 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:25:55.008272    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd4c48c_804d_4c71_975d_07954eea0274.slice/crio-8a7f9d36f12591df0d0b307f97aacab5c97841ef1c94f9565c580778640e594e WatchSource:0}: Error finding container 8a7f9d36f12591df0d0b307f97aacab5c97841ef1c94f9565c580778640e594e: Status 404 returned error can't find the container with id 8a7f9d36f12591df0d0b307f97aacab5c97841ef1c94f9565c580778640e594e
Nov 03 22:25:55.009949 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.009926    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"]
Nov 03 22:25:55.081746 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:25:55.081719    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa1f333_5c2e_46c6_acfe_09641db947ec.slice/crio-d7472e818e88abe2adfbb661cc916e18cf700d9810351ef1457f7c241aa8e0ec WatchSource:0}: Error finding container d7472e818e88abe2adfbb661cc916e18cf700d9810351ef1457f7c241aa8e0ec: Status 404 returned error can't find the container with id d7472e818e88abe2adfbb661cc916e18cf700d9810351ef1457f7c241aa8e0ec
Nov 03 22:25:55.083075 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.083053    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"]
Nov 03 22:25:55.249674 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:25:55.249629    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9e5c5d_7a5b_4d87_8710_030e9d175426.slice/crio-5a1b6031a766dc81f9642a024bd89ee83ed195afab1910637f62a87184cb804e WatchSource:0}: Error finding container 5a1b6031a766dc81f9642a024bd89ee83ed195afab1910637f62a87184cb804e: Status 404 returned error can't find the container with id 5a1b6031a766dc81f9642a024bd89ee83ed195afab1910637f62a87184cb804e
Nov 03 22:25:55.250071 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.250011    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"]
Nov 03 22:25:55.544238 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.544196    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr" event={"ID":"d159095a-05c8-4eeb-a501-c6ca0935c4a9","Type":"ContainerStarted","Data":"c766ac5be2f5e8959a4120f81ac0d5c571c04790388dcc81203edfa15e5a98fe"}
Nov 03 22:25:55.545602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.545575    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt" event={"ID":"6b9e5c5d-7a5b-4d87-8710-030e9d175426","Type":"ContainerStarted","Data":"5a1b6031a766dc81f9642a024bd89ee83ed195afab1910637f62a87184cb804e"}
Nov 03 22:25:55.546942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.546920    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s" event={"ID":"4cd4c48c-804d-4c71-975d-07954eea0274","Type":"ContainerStarted","Data":"8a7f9d36f12591df0d0b307f97aacab5c97841ef1c94f9565c580778640e594e"}
Nov 03 22:25:55.548221 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.548196    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk" event={"ID":"5aa1f333-5c2e-46c6-acfe-09641db947ec","Type":"ContainerStarted","Data":"d7472e818e88abe2adfbb661cc916e18cf700d9810351ef1457f7c241aa8e0ec"}
Nov 03 22:25:55.556189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.556170    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"]
Nov 03 22:25:55.561034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.561016    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.618127 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.618086    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-vpwbg\""
Nov 03 22:25:55.619454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.619428    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"]
Nov 03 22:25:55.654861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.654823    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.654979 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.654878    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.655056 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.655002    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s55nh\" (UniqueName: \"kubernetes.io/projected/fa311aa4-3004-4bef-bd02-08d2d278dbd0-kube-api-access-s55nh\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.755783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.755751    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.755958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.755835    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s55nh\" (UniqueName: \"kubernetes.io/projected/fa311aa4-3004-4bef-bd02-08d2d278dbd0-kube-api-access-s55nh\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.755958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.755877    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.756189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.756163    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.756271 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.756219    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.781677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.781650    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"]
Nov 03 22:25:55.785889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.785869    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.818945 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.818888    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s55nh\" (UniqueName: \"kubernetes.io/projected/fa311aa4-3004-4bef-bd02-08d2d278dbd0-kube-api-access-s55nh\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.842522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.842499    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-h5xqf\""
Nov 03 22:25:55.843825 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.843799    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"]
Nov 03 22:25:55.857047 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.856970    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.857047 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.857023    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgvhn\" (UniqueName: \"kubernetes.io/projected/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-kube-api-access-vgvhn\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.857189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.857161    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.878892 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.878862    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:25:55.958809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.958274    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.958809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.958325    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgvhn\" (UniqueName: \"kubernetes.io/projected/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-kube-api-access-vgvhn\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.958809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.958400    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.958809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.958743    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:55.958809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:55.958770    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:56.022252 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:56.022220    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgvhn\" (UniqueName: \"kubernetes.io/projected/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-kube-api-access-vgvhn\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:56.103377 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:56.103299    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:25:56.259798 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:56.258636    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"]
Nov 03 22:25:56.263599 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:25:56.263556    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa311aa4_3004_4bef_bd02_08d2d278dbd0.slice/crio-0e8251f228b3b8ecbe1d429901331da667dc83f8641f09aa8022fbe33ac91455 WatchSource:0}: Error finding container 0e8251f228b3b8ecbe1d429901331da667dc83f8641f09aa8022fbe33ac91455: Status 404 returned error can't find the container with id 0e8251f228b3b8ecbe1d429901331da667dc83f8641f09aa8022fbe33ac91455
Nov 03 22:25:56.503048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:56.502724    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"]
Nov 03 22:25:56.506405 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:25:56.506341    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a34f8bb_03ad_4f8d_99d6_b87e0679bd43.slice/crio-dc7a51f508bdca2e5fa3e96c7444302091822c5ba34e8da761d391064d378c3d WatchSource:0}: Error finding container dc7a51f508bdca2e5fa3e96c7444302091822c5ba34e8da761d391064d378c3d: Status 404 returned error can't find the container with id dc7a51f508bdca2e5fa3e96c7444302091822c5ba34e8da761d391064d378c3d
Nov 03 22:25:56.556485 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:56.556417    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc" event={"ID":"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43","Type":"ContainerStarted","Data":"dc7a51f508bdca2e5fa3e96c7444302091822c5ba34e8da761d391064d378c3d"}
Nov 03 22:25:56.559715 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:56.559688    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67" event={"ID":"fa311aa4-3004-4bef-bd02-08d2d278dbd0","Type":"ContainerStarted","Data":"0e8251f228b3b8ecbe1d429901331da667dc83f8641f09aa8022fbe33ac91455"}
Nov 03 22:25:57.153855 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:57.153741    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:57.357196 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:57.357029    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:58.553974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.553914    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:58.724645 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.724615    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"]
Nov 03 22:25:58.729726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.729643    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.784307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.784276    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.790888    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.790942    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.790961    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.791014    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.791032    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.791050    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.791269    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.791292    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htm9k\" (UniqueName: \"kubernetes.io/projected/9227810a-8c4b-41e8-a17b-1add6a0428e1-kube-api-access-htm9k\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.792372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.791323    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9227810a-8c4b-41e8-a17b-1add6a0428e1-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.808702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.808375    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:58.814091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.813980    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 22:25:58.814091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.814009    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 22:25:58.837579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.837326    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"]
Nov 03 22:25:58.837579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.837528    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-nxpxw\""
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891771    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891801    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891820    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891842    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891858    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htm9k\" (UniqueName: \"kubernetes.io/projected/9227810a-8c4b-41e8-a17b-1add6a0428e1-kube-api-access-htm9k\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891894    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9227810a-8c4b-41e8-a17b-1add6a0428e1-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891915    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891943    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.892307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.891960    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.894189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.893283    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.894189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.893543    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.894189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.893810    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.894189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.894118    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9227810a-8c4b-41e8-a17b-1add6a0428e1-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.894890 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.894845    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.896495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.896475    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.898484 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.898426    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.986239 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.986207    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9227810a-8c4b-41e8-a17b-1add6a0428e1-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:58.989203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:58.989136    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htm9k\" (UniqueName: \"kubernetes.io/projected/9227810a-8c4b-41e8-a17b-1add6a0428e1-kube-api-access-htm9k\") pod \"openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5\" (UID: \"9227810a-8c4b-41e8-a17b-1add6a0428e1\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:59.010602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:59.010574    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:25:59.048465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:59.048436    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:25:59.511587 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:59.511557    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"]
Nov 03 22:25:59.587657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:25:59.587563    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5" event={"ID":"9227810a-8c4b-41e8-a17b-1add6a0428e1","Type":"ContainerStarted","Data":"7e0c9b4bfba770d37c77d56188c71859a9b60a7f5c3559ffe7c2ab462829db52"}
Nov 03 22:26:02.574985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:02.574944    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:05.674726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:05.674694    5162 generic.go:356] "Generic (PLEG): container finished" podID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerID="944cba59910c4500944fca0d5193cbf460b0a2b25b40313d2ae5cf1fd0e1a880" exitCode=0
Nov 03 22:26:05.675249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:05.674739    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm" event={"ID":"5d3da421-41cc-4400-a8ab-2a7a14c91462","Type":"ContainerDied","Data":"944cba59910c4500944fca0d5193cbf460b0a2b25b40313d2ae5cf1fd0e1a880"}
Nov 03 22:26:06.687438 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:06.686652    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerID="f9aec982e92c8a956e11faacd16c6ece1bf138cfdb6e45b0d8da31c188d90912" exitCode=0
Nov 03 22:26:06.687438 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:06.686777    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc" event={"ID":"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43","Type":"ContainerDied","Data":"f9aec982e92c8a956e11faacd16c6ece1bf138cfdb6e45b0d8da31c188d90912"}
Nov 03 22:26:06.691360 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:06.689825    5162 generic.go:356] "Generic (PLEG): container finished" podID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerID="21fd2ff87c8befc68feb6e7835303d2a2a4a487b50ccbab5ebc11b32c04a14f8" exitCode=0
Nov 03 22:26:06.691360 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:06.689850    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67" event={"ID":"fa311aa4-3004-4bef-bd02-08d2d278dbd0","Type":"ContainerDied","Data":"21fd2ff87c8befc68feb6e7835303d2a2a4a487b50ccbab5ebc11b32c04a14f8"}
Nov 03 22:26:07.086378 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:07.086336    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:26:07.086530 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:07.086442    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:26:07.086530 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:07.086490    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:26:07.698726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:07.697742    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5" event={"ID":"9227810a-8c4b-41e8-a17b-1add6a0428e1","Type":"ContainerStarted","Data":"a4102268476ad4b79ffea5cdf5e622015e966255e6b96ad4f20dc2108bf2a940"}
Nov 03 22:26:07.700661 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:07.700631    5162 generic.go:356] "Generic (PLEG): container finished" podID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerID="5d28f916bd27088f5bb02ac770c950c114d48a280fab8ff580343783a77a13a5" exitCode=0
Nov 03 22:26:07.700791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:07.700694    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm" event={"ID":"5d3da421-41cc-4400-a8ab-2a7a14c91462","Type":"ContainerDied","Data":"5d28f916bd27088f5bb02ac770c950c114d48a280fab8ff580343783a77a13a5"}
Nov 03 22:26:07.825997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:07.825683    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5" podStartSLOduration=2.263240527 podStartE2EDuration="9.825663528s" totalImagesPullingTime="7.562423001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:07.823338148 +0000 UTC m=+171.088038161" watchObservedRunningTime="2026-11-03 22:26:07.825663528 +0000 UTC m=+171.090363543"
Nov 03 22:26:08.049216 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:08.049106    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:26:08.055485 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:08.055455    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:26:08.706994 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:08.706968    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:26:08.708123 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:08.708061    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5"
Nov 03 22:26:09.713907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.713006    5162 generic.go:356] "Generic (PLEG): container finished" podID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerID="01a0873967729c677f7abcf074e1accdfa2bfd87ccd57701ba1985358af96339" exitCode=0
Nov 03 22:26:09.713907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.713091    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk" event={"ID":"5aa1f333-5c2e-46c6-acfe-09641db947ec","Type":"ContainerDied","Data":"01a0873967729c677f7abcf074e1accdfa2bfd87ccd57701ba1985358af96339"}
Nov 03 22:26:09.720686 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.720470    5162 generic.go:356] "Generic (PLEG): container finished" podID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerID="c90695e25382e8050bd4d5648d735386d356a700fac59170a797c8879fd7e8b2" exitCode=0
Nov 03 22:26:09.720686 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.720599    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr" event={"ID":"d159095a-05c8-4eeb-a501-c6ca0935c4a9","Type":"ContainerDied","Data":"c90695e25382e8050bd4d5648d735386d356a700fac59170a797c8879fd7e8b2"}
Nov 03 22:26:09.723619 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.723594    5162 generic.go:356] "Generic (PLEG): container finished" podID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerID="673f7c4c806c42af15e20eee9c44092709bea29701bee144079329d345b7fec4" exitCode=0
Nov 03 22:26:09.723919 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.723863    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt" event={"ID":"6b9e5c5d-7a5b-4d87-8710-030e9d175426","Type":"ContainerDied","Data":"673f7c4c806c42af15e20eee9c44092709bea29701bee144079329d345b7fec4"}
Nov 03 22:26:09.726547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.726526    5162 generic.go:356] "Generic (PLEG): container finished" podID="4cd4c48c-804d-4c71-975d-07954eea0274" containerID="bee397688716ff7058bfd227273eabe692c46e5a5cf46d7bbaaf316a38856360" exitCode=0
Nov 03 22:26:09.726635 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:09.726610    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s" event={"ID":"4cd4c48c-804d-4c71-975d-07954eea0274","Type":"ContainerDied","Data":"bee397688716ff7058bfd227273eabe692c46e5a5cf46d7bbaaf316a38856360"}
Nov 03 22:26:10.093712 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:26:10.093632    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:26:11.737937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.737910    5162 generic.go:356] "Generic (PLEG): container finished" podID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerID="b747fab159c4830eee8de40e27fee53a9c7eba9090ce5fbd165b4975e10d61e2" exitCode=0
Nov 03 22:26:11.738288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.737997    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr" event={"ID":"d159095a-05c8-4eeb-a501-c6ca0935c4a9","Type":"ContainerDied","Data":"b747fab159c4830eee8de40e27fee53a9c7eba9090ce5fbd165b4975e10d61e2"}
Nov 03 22:26:11.739955 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.739927    5162 generic.go:356] "Generic (PLEG): container finished" podID="4cd4c48c-804d-4c71-975d-07954eea0274" containerID="def68c15ab2b32352ad2439a5a7f912489c914e1dee79d6e1c8a0e710a9dd60e" exitCode=0
Nov 03 22:26:11.740045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.740018    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s" event={"ID":"4cd4c48c-804d-4c71-975d-07954eea0274","Type":"ContainerDied","Data":"def68c15ab2b32352ad2439a5a7f912489c914e1dee79d6e1c8a0e710a9dd60e"}
Nov 03 22:26:11.742374 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.742351    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerID="28090e21f940249c39341327ce7a406537ff62898cfe26559c8051cdba32eb03" exitCode=0
Nov 03 22:26:11.742478 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.742418    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc" event={"ID":"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43","Type":"ContainerDied","Data":"28090e21f940249c39341327ce7a406537ff62898cfe26559c8051cdba32eb03"}
Nov 03 22:26:11.744677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.744655    5162 generic.go:356] "Generic (PLEG): container finished" podID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerID="8e4bd77ebe4b0b899ead43aec318cd6dbb8288241829180cd9c635d5ea4d112e" exitCode=0
Nov 03 22:26:11.744782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:11.744718    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67" event={"ID":"fa311aa4-3004-4bef-bd02-08d2d278dbd0","Type":"ContainerDied","Data":"8e4bd77ebe4b0b899ead43aec318cd6dbb8288241829180cd9c635d5ea4d112e"}
Nov 03 22:26:12.751418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:12.751280    5162 generic.go:356] "Generic (PLEG): container finished" podID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerID="83e3e923d8de02799eae7d1e82f56e88b607b5c100f569332430c31aefc9df41" exitCode=0
Nov 03 22:26:12.751418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:12.751381    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk" event={"ID":"5aa1f333-5c2e-46c6-acfe-09641db947ec","Type":"ContainerDied","Data":"83e3e923d8de02799eae7d1e82f56e88b607b5c100f569332430c31aefc9df41"}
Nov 03 22:26:12.753504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:12.753449    5162 generic.go:356] "Generic (PLEG): container finished" podID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerID="af4117931d83d10ea09eb527476e250674929c63dcc979502c6701f75bcba367" exitCode=0
Nov 03 22:26:12.753504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:12.753484    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt" event={"ID":"6b9e5c5d-7a5b-4d87-8710-030e9d175426","Type":"ContainerDied","Data":"af4117931d83d10ea09eb527476e250674929c63dcc979502c6701f75bcba367"}
Nov 03 22:26:22.836290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:22.836254    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm" event={"ID":"5d3da421-41cc-4400-a8ab-2a7a14c91462","Type":"ContainerStarted","Data":"f32fbd8d55dc98cfd36e97625996a3ab29759f6445a37745ba5be96f241c51ab"}
Nov 03 22:26:22.961250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:22.961185    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm" podStartSLOduration=2.652442967 podStartE2EDuration="30.961164642s" totalImagesPullingTime="28.308721675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:25:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:22.960444568 +0000 UTC m=+186.225144596" watchObservedRunningTime="2026-11-03 22:26:22.961164642 +0000 UTC m=+186.225864654"
Nov 03 22:26:23.846628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:23.846061    5162 generic.go:356] "Generic (PLEG): container finished" podID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerID="f32fbd8d55dc98cfd36e97625996a3ab29759f6445a37745ba5be96f241c51ab" exitCode=0
Nov 03 22:26:23.846628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:23.846188    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm" event={"ID":"5d3da421-41cc-4400-a8ab-2a7a14c91462","Type":"ContainerDied","Data":"f32fbd8d55dc98cfd36e97625996a3ab29759f6445a37745ba5be96f241c51ab"}
Nov 03 22:26:25.070736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.070701    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:26:25.198915 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.198876    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d3da421-41cc-4400-a8ab-2a7a14c91462-kube-api-access-mk5dh\" (UniqueName: \"kubernetes.io/projected/5d3da421-41cc-4400-a8ab-2a7a14c91462-kube-api-access-mk5dh\") pod \"5d3da421-41cc-4400-a8ab-2a7a14c91462\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") "
Nov 03 22:26:25.199105 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.198948    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-bundle\") pod \"5d3da421-41cc-4400-a8ab-2a7a14c91462\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") "
Nov 03 22:26:25.199105 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.199045    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-util\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-util\") pod \"5d3da421-41cc-4400-a8ab-2a7a14c91462\" (UID: \"5d3da421-41cc-4400-a8ab-2a7a14c91462\") "
Nov 03 22:26:25.203143 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.203091    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-bundle" pod "5d3da421-41cc-4400-a8ab-2a7a14c91462" (UID: "5d3da421-41cc-4400-a8ab-2a7a14c91462"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:25.205760 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.205709    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3da421-41cc-4400-a8ab-2a7a14c91462-kube-api-access-mk5dh" pod "5d3da421-41cc-4400-a8ab-2a7a14c91462" (UID: "5d3da421-41cc-4400-a8ab-2a7a14c91462"). InnerVolumeSpecName "kube-api-access-mk5dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:26:25.220901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.220849    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-util" pod "5d3da421-41cc-4400-a8ab-2a7a14c91462" (UID: "5d3da421-41cc-4400-a8ab-2a7a14c91462"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:25.300276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.300124    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk5dh\" (UniqueName: \"kubernetes.io/projected/5d3da421-41cc-4400-a8ab-2a7a14c91462-kube-api-access-mk5dh\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:25.300276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.300156    5162 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:25.300276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.300174    5162 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3da421-41cc-4400-a8ab-2a7a14c91462-util\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:25.861508 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.861469    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm" event={"ID":"5d3da421-41cc-4400-a8ab-2a7a14c91462","Type":"ContainerDied","Data":"1f691dcbe890c1c3b21e59eaca57cc1025a00167679b473df929a9ceabf6ff2f"}
Nov 03 22:26:25.861703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.861532    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f691dcbe890c1c3b21e59eaca57cc1025a00167679b473df929a9ceabf6ff2f"
Nov 03 22:26:25.861703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.861626    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rgkfm"
Nov 03 22:26:25.876467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:25.876436    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:26.869963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:26.869469    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr" event={"ID":"d159095a-05c8-4eeb-a501-c6ca0935c4a9","Type":"ContainerStarted","Data":"3c3bf348c6f37f7c386a8c9ae120cdddde0df6c80d839d9d59f7196cba711a0f"}
Nov 03 22:26:26.872468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:26.872440    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s" event={"ID":"4cd4c48c-804d-4c71-975d-07954eea0274","Type":"ContainerStarted","Data":"df1461ffc24c17edb707f365adca38bbb536c9b3c47326ec1a18687a602230aa"}
Nov 03 22:26:26.874940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:26.874910    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc" event={"ID":"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43","Type":"ContainerStarted","Data":"59f1d4bf5eda334bed1c8e617c3dd9b5c44fdfb6ac5a77c5394dad4676eccb99"}
Nov 03 22:26:26.879259 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:26.879232    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67" event={"ID":"fa311aa4-3004-4bef-bd02-08d2d278dbd0","Type":"ContainerStarted","Data":"0eadc938e67d749668d8521e3ecfc30bef95b054d1ecf766c3eb7cd443a4ea50"}
Nov 03 22:26:27.022871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.022781    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr" podStartSLOduration=2.143348733 podStartE2EDuration="33.022767672s" totalImagesPullingTime="30.879418939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:25:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:27.022437466 +0000 UTC m=+190.287137478" watchObservedRunningTime="2026-11-03 22:26:27.022767672 +0000 UTC m=+190.287467674"
Nov 03 22:26:27.144694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.144643    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s" podStartSLOduration=2.426667009 podStartE2EDuration="33.144624737s" totalImagesPullingTime="30.717957728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:25:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:27.143246652 +0000 UTC m=+190.407946666" watchObservedRunningTime="2026-11-03 22:26:27.144624737 +0000 UTC m=+190.409324749"
Nov 03 22:26:27.266106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.266041    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67" podStartSLOduration=3.331335643 podStartE2EDuration="32.266019257s" totalImagesPullingTime="28.934683614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:25:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:27.264668499 +0000 UTC m=+190.529368511" watchObservedRunningTime="2026-11-03 22:26:27.266019257 +0000 UTC m=+190.530719266"
Nov 03 22:26:27.386355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.386258    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc" podStartSLOduration=3.7474550779999998 podStartE2EDuration="32.386243876s" totalImagesPullingTime="28.638788798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:25:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:27.385826445 +0000 UTC m=+190.650526453" watchObservedRunningTime="2026-11-03 22:26:27.386243876 +0000 UTC m=+190.650943887"
Nov 03 22:26:27.885736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.885708    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk" event={"ID":"5aa1f333-5c2e-46c6-acfe-09641db947ec","Type":"ContainerStarted","Data":"2f1b118dee527fef2b9e18a943278ba3f4268570a771e4e11136c3d89e828a0b"}
Nov 03 22:26:27.887661 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.887638    5162 generic.go:356] "Generic (PLEG): container finished" podID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerID="3c3bf348c6f37f7c386a8c9ae120cdddde0df6c80d839d9d59f7196cba711a0f" exitCode=0
Nov 03 22:26:27.887776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.887729    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr" event={"ID":"d159095a-05c8-4eeb-a501-c6ca0935c4a9","Type":"ContainerDied","Data":"3c3bf348c6f37f7c386a8c9ae120cdddde0df6c80d839d9d59f7196cba711a0f"}
Nov 03 22:26:27.889702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.889680    5162 generic.go:356] "Generic (PLEG): container finished" podID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerID="2b2e3f32865e4ff54a1000946edb621bd0f7e38060955242b454c8787173d0b6" exitCode=0
Nov 03 22:26:27.889791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.889761    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt" event={"ID":"6b9e5c5d-7a5b-4d87-8710-030e9d175426","Type":"ContainerDied","Data":"2b2e3f32865e4ff54a1000946edb621bd0f7e38060955242b454c8787173d0b6"}
Nov 03 22:26:27.891630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.891608    5162 generic.go:356] "Generic (PLEG): container finished" podID="4cd4c48c-804d-4c71-975d-07954eea0274" containerID="df1461ffc24c17edb707f365adca38bbb536c9b3c47326ec1a18687a602230aa" exitCode=0
Nov 03 22:26:27.891736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.891693    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s" event={"ID":"4cd4c48c-804d-4c71-975d-07954eea0274","Type":"ContainerDied","Data":"df1461ffc24c17edb707f365adca38bbb536c9b3c47326ec1a18687a602230aa"}
Nov 03 22:26:27.893492 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.893473    5162 generic.go:356] "Generic (PLEG): container finished" podID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerID="59f1d4bf5eda334bed1c8e617c3dd9b5c44fdfb6ac5a77c5394dad4676eccb99" exitCode=0
Nov 03 22:26:27.893582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.893537    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc" event={"ID":"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43","Type":"ContainerDied","Data":"59f1d4bf5eda334bed1c8e617c3dd9b5c44fdfb6ac5a77c5394dad4676eccb99"}
Nov 03 22:26:27.895489 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.895465    5162 generic.go:356] "Generic (PLEG): container finished" podID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerID="0eadc938e67d749668d8521e3ecfc30bef95b054d1ecf766c3eb7cd443a4ea50" exitCode=0
Nov 03 22:26:27.895560 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:27.895494    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67" event={"ID":"fa311aa4-3004-4bef-bd02-08d2d278dbd0","Type":"ContainerDied","Data":"0eadc938e67d749668d8521e3ecfc30bef95b054d1ecf766c3eb7cd443a4ea50"}
Nov 03 22:26:28.007351 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:28.007296    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk" podStartSLOduration=3.4184013650000002 podStartE2EDuration="34.007281387s" totalImagesPullingTime="30.588880022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:25:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:28.006961478 +0000 UTC m=+191.271661490" watchObservedRunningTime="2026-11-03 22:26:28.007281387 +0000 UTC m=+191.271981398"
Nov 03 22:26:28.760793 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:28.760764    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:28.901377 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:28.901342    5162 generic.go:356] "Generic (PLEG): container finished" podID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerID="2f1b118dee527fef2b9e18a943278ba3f4268570a771e4e11136c3d89e828a0b" exitCode=0
Nov 03 22:26:28.901857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:28.901440    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk" event={"ID":"5aa1f333-5c2e-46c6-acfe-09641db947ec","Type":"ContainerDied","Data":"2f1b118dee527fef2b9e18a943278ba3f4268570a771e4e11136c3d89e828a0b"}
Nov 03 22:26:29.046633 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.046604    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:26:29.119502 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.119479    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:26:29.122722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.122700    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:26:29.126044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.126026    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:26:29.129049 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.129031    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:26:29.133141 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.133117    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-bundle\") pod \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") "
Nov 03 22:26:29.133278 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.133155    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-util\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-util\") pod \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") "
Nov 03 22:26:29.133278 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.133236    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b9e5c5d-7a5b-4d87-8710-030e9d175426-kube-api-access-6t96s\" (UniqueName: \"kubernetes.io/projected/6b9e5c5d-7a5b-4d87-8710-030e9d175426-kube-api-access-6t96s\") pod \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\" (UID: \"6b9e5c5d-7a5b-4d87-8710-030e9d175426\") "
Nov 03 22:26:29.133611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.133589    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-bundle" pod "6b9e5c5d-7a5b-4d87-8710-030e9d175426" (UID: "6b9e5c5d-7a5b-4d87-8710-030e9d175426"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.135147 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.135128    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9e5c5d-7a5b-4d87-8710-030e9d175426-kube-api-access-6t96s" pod "6b9e5c5d-7a5b-4d87-8710-030e9d175426" (UID: "6b9e5c5d-7a5b-4d87-8710-030e9d175426"). InnerVolumeSpecName "kube-api-access-6t96s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:26:29.138194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.138175    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-util" pod "6b9e5c5d-7a5b-4d87-8710-030e9d175426" (UID: "6b9e5c5d-7a5b-4d87-8710-030e9d175426"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.234489 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234466    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-util\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-util\") pod \"4cd4c48c-804d-4c71-975d-07954eea0274\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") "
Nov 03 22:26:29.234644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234534    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-bundle\") pod \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") "
Nov 03 22:26:29.234644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234564    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d159095a-05c8-4eeb-a501-c6ca0935c4a9-kube-api-access-2q58x\" (UniqueName: \"kubernetes.io/projected/d159095a-05c8-4eeb-a501-c6ca0935c4a9-kube-api-access-2q58x\") pod \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") "
Nov 03 22:26:29.234644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234593    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-bundle\") pod \"4cd4c48c-804d-4c71-975d-07954eea0274\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") "
Nov 03 22:26:29.234644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234623    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa311aa4-3004-4bef-bd02-08d2d278dbd0-kube-api-access-s55nh\" (UniqueName: \"kubernetes.io/projected/fa311aa4-3004-4bef-bd02-08d2d278dbd0-kube-api-access-s55nh\") pod \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") "
Nov 03 22:26:29.234866 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234654    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-bundle\") pod \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") "
Nov 03 22:26:29.234866 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234682    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-util\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-util\") pod \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\" (UID: \"fa311aa4-3004-4bef-bd02-08d2d278dbd0\") "
Nov 03 22:26:29.234866 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234761    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-util\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-util\") pod \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") "
Nov 03 22:26:29.234866 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.234855    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-kube-api-access-vgvhn\" (UniqueName: \"kubernetes.io/projected/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-kube-api-access-vgvhn\") pod \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") "
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.235149    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-bundle" pod "fa311aa4-3004-4bef-bd02-08d2d278dbd0" (UID: "fa311aa4-3004-4bef-bd02-08d2d278dbd0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.235171    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-bundle" pod "4cd4c48c-804d-4c71-975d-07954eea0274" (UID: "4cd4c48c-804d-4c71-975d-07954eea0274"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.235610    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-bundle" pod "1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" (UID: "1a34f8bb-03ad-4f8d-99d6-b87e0679bd43"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.235693    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-bundle\") pod \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\" (UID: \"d159095a-05c8-4eeb-a501-c6ca0935c4a9\") "
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.235731    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cd4c48c-804d-4c71-975d-07954eea0274-kube-api-access-mzp7b\" (UniqueName: \"kubernetes.io/projected/4cd4c48c-804d-4c71-975d-07954eea0274-kube-api-access-mzp7b\") pod \"4cd4c48c-804d-4c71-975d-07954eea0274\" (UID: \"4cd4c48c-804d-4c71-975d-07954eea0274\") "
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.235761    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-util\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-util\") pod \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\" (UID: \"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43\") "
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.236017    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t96s\" (UniqueName: \"kubernetes.io/projected/6b9e5c5d-7a5b-4d87-8710-030e9d175426-kube-api-access-6t96s\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.236034    5162 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.236050    5162 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.236063    5162 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.236073    5162 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.236085    5162 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b9e5c5d-7a5b-4d87-8710-030e9d175426-util\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.236790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.236774    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-bundle" pod "d159095a-05c8-4eeb-a501-c6ca0935c4a9" (UID: "d159095a-05c8-4eeb-a501-c6ca0935c4a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.237814 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.237733    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d159095a-05c8-4eeb-a501-c6ca0935c4a9-kube-api-access-2q58x" pod "d159095a-05c8-4eeb-a501-c6ca0935c4a9" (UID: "d159095a-05c8-4eeb-a501-c6ca0935c4a9"). InnerVolumeSpecName "kube-api-access-2q58x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:26:29.237814 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.237774    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-kube-api-access-vgvhn" pod "1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" (UID: "1a34f8bb-03ad-4f8d-99d6-b87e0679bd43"). InnerVolumeSpecName "kube-api-access-vgvhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:26:29.238035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.238016    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa311aa4-3004-4bef-bd02-08d2d278dbd0-kube-api-access-s55nh" pod "fa311aa4-3004-4bef-bd02-08d2d278dbd0" (UID: "fa311aa4-3004-4bef-bd02-08d2d278dbd0"). InnerVolumeSpecName "kube-api-access-s55nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:26:29.238574 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.238543    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cd4c48c-804d-4c71-975d-07954eea0274-kube-api-access-mzp7b" pod "4cd4c48c-804d-4c71-975d-07954eea0274" (UID: "4cd4c48c-804d-4c71-975d-07954eea0274"). InnerVolumeSpecName "kube-api-access-mzp7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:26:29.242029 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.241999    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-util" pod "4cd4c48c-804d-4c71-975d-07954eea0274" (UID: "4cd4c48c-804d-4c71-975d-07954eea0274"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.242408 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.242375    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-util" pod "d159095a-05c8-4eeb-a501-c6ca0935c4a9" (UID: "d159095a-05c8-4eeb-a501-c6ca0935c4a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.243422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.243385    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-util" pod "fa311aa4-3004-4bef-bd02-08d2d278dbd0" (UID: "fa311aa4-3004-4bef-bd02-08d2d278dbd0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.244016 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.244000    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-util" pod "1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" (UID: "1a34f8bb-03ad-4f8d-99d6-b87e0679bd43"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:29.337057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.336996    5162 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-util\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337018    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgvhn\" (UniqueName: \"kubernetes.io/projected/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-kube-api-access-vgvhn\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337028    5162 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d159095a-05c8-4eeb-a501-c6ca0935c4a9-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337039    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzp7b\" (UniqueName: \"kubernetes.io/projected/4cd4c48c-804d-4c71-975d-07954eea0274-kube-api-access-mzp7b\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337049    5162 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34f8bb-03ad-4f8d-99d6-b87e0679bd43-util\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337056    5162 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cd4c48c-804d-4c71-975d-07954eea0274-util\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337299 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337064    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q58x\" (UniqueName: \"kubernetes.io/projected/d159095a-05c8-4eeb-a501-c6ca0935c4a9-kube-api-access-2q58x\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337299 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337073    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s55nh\" (UniqueName: \"kubernetes.io/projected/fa311aa4-3004-4bef-bd02-08d2d278dbd0-kube-api-access-s55nh\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.337299 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.337082    5162 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa311aa4-3004-4bef-bd02-08d2d278dbd0-util\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:29.906992 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.906968    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt"
Nov 03 22:26:29.907449 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.906961    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mrbrt" event={"ID":"6b9e5c5d-7a5b-4d87-8710-030e9d175426","Type":"ContainerDied","Data":"5a1b6031a766dc81f9642a024bd89ee83ed195afab1910637f62a87184cb804e"}
Nov 03 22:26:29.907449 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.907092    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a1b6031a766dc81f9642a024bd89ee83ed195afab1910637f62a87184cb804e"
Nov 03 22:26:29.908884 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.908862    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s" event={"ID":"4cd4c48c-804d-4c71-975d-07954eea0274","Type":"ContainerDied","Data":"8a7f9d36f12591df0d0b307f97aacab5c97841ef1c94f9565c580778640e594e"}
Nov 03 22:26:29.908884 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.908879    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mg44s"
Nov 03 22:26:29.909025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.908902    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a7f9d36f12591df0d0b307f97aacab5c97841ef1c94f9565c580778640e594e"
Nov 03 22:26:29.910738 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.910722    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc"
Nov 03 22:26:29.910836 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.910723    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c9q4jc" event={"ID":"1a34f8bb-03ad-4f8d-99d6-b87e0679bd43","Type":"ContainerDied","Data":"dc7a51f508bdca2e5fa3e96c7444302091822c5ba34e8da761d391064d378c3d"}
Nov 03 22:26:29.910899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.910841    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc7a51f508bdca2e5fa3e96c7444302091822c5ba34e8da761d391064d378c3d"
Nov 03 22:26:29.912607 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.912584    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67"
Nov 03 22:26:29.912703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.912621    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdv67" event={"ID":"fa311aa4-3004-4bef-bd02-08d2d278dbd0","Type":"ContainerDied","Data":"0e8251f228b3b8ecbe1d429901331da667dc83f8641f09aa8022fbe33ac91455"}
Nov 03 22:26:29.912703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.912656    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e8251f228b3b8ecbe1d429901331da667dc83f8641f09aa8022fbe33ac91455"
Nov 03 22:26:29.914768 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.914745    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr" event={"ID":"d159095a-05c8-4eeb-a501-c6ca0935c4a9","Type":"ContainerDied","Data":"c766ac5be2f5e8959a4120f81ac0d5c571c04790388dcc81203edfa15e5a98fe"}
Nov 03 22:26:29.914853 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.914780    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c766ac5be2f5e8959a4120f81ac0d5c571c04790388dcc81203edfa15e5a98fe"
Nov 03 22:26:29.914853 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:29.914798    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxxgxr"
Nov 03 22:26:30.027424 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.027284    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:26:30.145048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.145018    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-bundle\") pod \"5aa1f333-5c2e-46c6-acfe-09641db947ec\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") "
Nov 03 22:26:30.145191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.145135    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5aa1f333-5c2e-46c6-acfe-09641db947ec-kube-api-access-j5nbz\" (UniqueName: \"kubernetes.io/projected/5aa1f333-5c2e-46c6-acfe-09641db947ec-kube-api-access-j5nbz\") pod \"5aa1f333-5c2e-46c6-acfe-09641db947ec\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") "
Nov 03 22:26:30.145191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.145186    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-util\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-util\") pod \"5aa1f333-5c2e-46c6-acfe-09641db947ec\" (UID: \"5aa1f333-5c2e-46c6-acfe-09641db947ec\") "
Nov 03 22:26:30.145830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.145795    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-bundle" pod "5aa1f333-5c2e-46c6-acfe-09641db947ec" (UID: "5aa1f333-5c2e-46c6-acfe-09641db947ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:30.147220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.147197    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aa1f333-5c2e-46c6-acfe-09641db947ec-kube-api-access-j5nbz" pod "5aa1f333-5c2e-46c6-acfe-09641db947ec" (UID: "5aa1f333-5c2e-46c6-acfe-09641db947ec"). InnerVolumeSpecName "kube-api-access-j5nbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:26:30.152766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.152742    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-util" pod "5aa1f333-5c2e-46c6-acfe-09641db947ec" (UID: "5aa1f333-5c2e-46c6-acfe-09641db947ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:26:30.246522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.246496    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5nbz\" (UniqueName: \"kubernetes.io/projected/5aa1f333-5c2e-46c6-acfe-09641db947ec-kube-api-access-j5nbz\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:30.246522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.246518    5162 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-util\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:30.246743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.246528    5162 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5aa1f333-5c2e-46c6-acfe-09641db947ec-bundle\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:26:30.317371 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.317338    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:30.555908 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.555826    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:30.921152 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.921119    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk" event={"ID":"5aa1f333-5c2e-46c6-acfe-09641db947ec","Type":"ContainerDied","Data":"d7472e818e88abe2adfbb661cc916e18cf700d9810351ef1457f7c241aa8e0ec"}
Nov 03 22:26:30.921152 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.921144    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6frk"
Nov 03 22:26:30.921604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:30.921170    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7472e818e88abe2adfbb661cc916e18cf700d9810351ef1457f7c241aa8e0ec"
Nov 03 22:26:31.007072 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.007034    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:31.131964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.131868    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"]
Nov 03 22:26:31.132211 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132199    5162 cpu_manager.go:414] "containerMap: removing container" podUID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerName="util"
Nov 03 22:26:31.132258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132212    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerName="util"
Nov 03 22:26:31.132258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132224    5162 cpu_manager.go:414] "containerMap: removing container" podUID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerName="extract"
Nov 03 22:26:31.132258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132230    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerName="extract"
Nov 03 22:26:31.132258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132241    5162 cpu_manager.go:414] "containerMap: removing container" podUID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerName="util"
Nov 03 22:26:31.132258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132247    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerName="util"
Nov 03 22:26:31.132258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132255    5162 cpu_manager.go:414] "containerMap: removing container" podUID="4cd4c48c-804d-4c71-975d-07954eea0274" containerName="pull"
Nov 03 22:26:31.132258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132259    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cd4c48c-804d-4c71-975d-07954eea0274" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132268    5162 cpu_manager.go:414] "containerMap: removing container" podUID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132273    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132280    5162 cpu_manager.go:414] "containerMap: removing container" podUID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132285    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132293    5162 cpu_manager.go:414] "containerMap: removing container" podUID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132297    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132306    5162 cpu_manager.go:414] "containerMap: removing container" podUID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132311    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132316    5162 cpu_manager.go:414] "containerMap: removing container" podUID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132321    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132329    5162 cpu_manager.go:414] "containerMap: removing container" podUID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132333    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132340    5162 cpu_manager.go:414] "containerMap: removing container" podUID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132344    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132355    5162 cpu_manager.go:414] "containerMap: removing container" podUID="4cd4c48c-804d-4c71-975d-07954eea0274" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132360    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cd4c48c-804d-4c71-975d-07954eea0274" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132367    5162 cpu_manager.go:414] "containerMap: removing container" podUID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132372    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132419    5162 cpu_manager.go:414] "containerMap: removing container" podUID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132426    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132436    5162 cpu_manager.go:414] "containerMap: removing container" podUID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132443    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerName="extract"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132457    5162 cpu_manager.go:414] "containerMap: removing container" podUID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132464    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132470    5162 cpu_manager.go:414] "containerMap: removing container" podUID="4cd4c48c-804d-4c71-975d-07954eea0274" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132475    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cd4c48c-804d-4c71-975d-07954eea0274" containerName="util"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132482    5162 cpu_manager.go:414] "containerMap: removing container" podUID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerName="pull"
Nov 03 22:26:31.132477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132488    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerName="pull"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132494    5162 cpu_manager.go:414] "containerMap: removing container" podUID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132499    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132506    5162 cpu_manager.go:414] "containerMap: removing container" podUID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerName="util"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132510    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerName="util"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132515    5162 cpu_manager.go:414] "containerMap: removing container" podUID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerName="pull"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132519    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerName="pull"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132565    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cd4c48c-804d-4c71-975d-07954eea0274" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132572    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b9e5c5d-7a5b-4d87-8710-030e9d175426" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132578    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="d159095a-05c8-4eeb-a501-c6ca0935c4a9" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132584    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a34f8bb-03ad-4f8d-99d6-b87e0679bd43" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132591    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d3da421-41cc-4400-a8ab-2a7a14c91462" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132598    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa311aa4-3004-4bef-bd02-08d2d278dbd0" containerName="extract"
Nov 03 22:26:31.133226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.132603    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="5aa1f333-5c2e-46c6-acfe-09641db947ec" containerName="extract"
Nov 03 22:26:31.135306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.135288    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.191106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.191071    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 03 22:26:31.191106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.191090    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 03 22:26:31.191324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.191122    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 03 22:26:31.191497 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.191482    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 03 22:26:31.191708 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.191696    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-6xv29\""
Nov 03 22:26:31.191951 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.191934    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 03 22:26:31.245052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.245016    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"]
Nov 03 22:26:31.256221 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.256190    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8964d89-365b-4c6b-9b60-573014ec8551-manifests\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.256370 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.256239    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.256448 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.256422    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfzp4\" (UniqueName: \"kubernetes.io/projected/b8964d89-365b-4c6b-9b60-573014ec8551-kube-api-access-mfzp4\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.256501 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.256474    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8964d89-365b-4c6b-9b60-573014ec8551-tmp\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.256541 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.256508    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.256571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.256537    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-metrics-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.285768 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.285731    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:31.358027 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.357981    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfzp4\" (UniqueName: \"kubernetes.io/projected/b8964d89-365b-4c6b-9b60-573014ec8551-kube-api-access-mfzp4\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.358027 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.358030    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8964d89-365b-4c6b-9b60-573014ec8551-tmp\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.358268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.358051    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.358268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.358163    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-metrics-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.358268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.358245    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8964d89-365b-4c6b-9b60-573014ec8551-manifests\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.358383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.358301    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.358649 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.358622    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8964d89-365b-4c6b-9b60-573014ec8551-manifests\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.360749 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.360723    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-metrics-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.360749 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.360748    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.360887 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.360767    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8964d89-365b-4c6b-9b60-573014ec8551-tmp\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.361085 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.361063    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8964d89-365b-4c6b-9b60-573014ec8551-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.421588 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.421559    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfzp4\" (UniqueName: \"kubernetes.io/projected/b8964d89-365b-4c6b-9b60-573014ec8551-kube-api-access-mfzp4\") pod \"opendatahub-operator-controller-manager-8bc866f6-qrjrh\" (UID: \"b8964d89-365b-4c6b-9b60-573014ec8551\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.448659 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.448623    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:31.495052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.495011    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:31.697714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.697628    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:31.933972 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:26:31.933941    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8964d89_365b_4c6b_9b60_573014ec8551.slice/crio-cbb5370eeea385121f68ea289ae793b894e276c8ac9590517cdb681b393701f6 WatchSource:0}: Error finding container cbb5370eeea385121f68ea289ae793b894e276c8ac9590517cdb681b393701f6: Status 404 returned error can't find the container with id cbb5370eeea385121f68ea289ae793b894e276c8ac9590517cdb681b393701f6
Nov 03 22:26:31.934413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:31.934240    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"]
Nov 03 22:26:32.319493 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:32.319458    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:32.939027 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:32.938987    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh" event={"ID":"b8964d89-365b-4c6b-9b60-573014ec8551","Type":"ContainerStarted","Data":"cbb5370eeea385121f68ea289ae793b894e276c8ac9590517cdb681b393701f6"}
Nov 03 22:26:33.417750 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:33.417670    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:33.920564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:33.920529    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:34.127842 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:34.127763    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:34.583209 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:34.583168    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:34.843430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:34.843334    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:35.955009 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:35.954971    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh" event={"ID":"b8964d89-365b-4c6b-9b60-573014ec8551","Type":"ContainerStarted","Data":"10fbfda2d9edff64042e7effa8d74f2f3f43c1de5ef73db8f4fac9b5231856ce"}
Nov 03 22:26:36.330973 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.330935    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"]
Nov 03 22:26:36.333183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.333169    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.390070 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.390050    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 03 22:26:36.390288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.390271    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:26:36.403576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.403560    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq6c8\" (UniqueName: \"kubernetes.io/projected/42413f4c-4836-4405-a2ce-b4b3d65f0ddf-kube-api-access-lq6c8\") pod \"jobset-operator-674cd59dc5-phj7s\" (UID: \"42413f4c-4836-4405-a2ce-b4b3d65f0ddf\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.403653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.403610    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42413f4c-4836-4405-a2ce-b4b3d65f0ddf-tmp\") pod \"jobset-operator-674cd59dc5-phj7s\" (UID: \"42413f4c-4836-4405-a2ce-b4b3d65f0ddf\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.419734 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.419711    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"]
Nov 03 22:26:36.449505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.449484    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-hch7p\""
Nov 03 22:26:36.504717 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.504654    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42413f4c-4836-4405-a2ce-b4b3d65f0ddf-tmp\") pod \"jobset-operator-674cd59dc5-phj7s\" (UID: \"42413f4c-4836-4405-a2ce-b4b3d65f0ddf\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.504717 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.504706    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq6c8\" (UniqueName: \"kubernetes.io/projected/42413f4c-4836-4405-a2ce-b4b3d65f0ddf-kube-api-access-lq6c8\") pod \"jobset-operator-674cd59dc5-phj7s\" (UID: \"42413f4c-4836-4405-a2ce-b4b3d65f0ddf\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.505021 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.505003    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42413f4c-4836-4405-a2ce-b4b3d65f0ddf-tmp\") pod \"jobset-operator-674cd59dc5-phj7s\" (UID: \"42413f4c-4836-4405-a2ce-b4b3d65f0ddf\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.567553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.567524    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq6c8\" (UniqueName: \"kubernetes.io/projected/42413f4c-4836-4405-a2ce-b4b3d65f0ddf-kube-api-access-lq6c8\") pod \"jobset-operator-674cd59dc5-phj7s\" (UID: \"42413f4c-4836-4405-a2ce-b4b3d65f0ddf\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.651827 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.651788    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"
Nov 03 22:26:36.960007 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.959972    5162 generic.go:356] "Generic (PLEG): container finished" podID="b8964d89-365b-4c6b-9b60-573014ec8551" containerID="10fbfda2d9edff64042e7effa8d74f2f3f43c1de5ef73db8f4fac9b5231856ce" exitCode=0
Nov 03 22:26:36.960452 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:36.960063    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh" event={"ID":"b8964d89-365b-4c6b-9b60-573014ec8551","Type":"ContainerDied","Data":"10fbfda2d9edff64042e7effa8d74f2f3f43c1de5ef73db8f4fac9b5231856ce"}
Nov 03 22:26:37.018656 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:26:37.018630    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42413f4c_4836_4405_a2ce_b4b3d65f0ddf.slice/crio-7b6e5bce3e2790f076f24073a9cbae69d24d57954f7452f0e02d7e7a964f19c9 WatchSource:0}: Error finding container 7b6e5bce3e2790f076f24073a9cbae69d24d57954f7452f0e02d7e7a964f19c9: Status 404 returned error can't find the container with id 7b6e5bce3e2790f076f24073a9cbae69d24d57954f7452f0e02d7e7a964f19c9
Nov 03 22:26:37.021074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:37.021057    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s"]
Nov 03 22:26:37.966983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:37.966948    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s" event={"ID":"42413f4c-4836-4405-a2ce-b4b3d65f0ddf","Type":"ContainerStarted","Data":"7b6e5bce3e2790f076f24073a9cbae69d24d57954f7452f0e02d7e7a964f19c9"}
Nov 03 22:26:37.969128 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:37.969102    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh" event={"ID":"b8964d89-365b-4c6b-9b60-573014ec8551","Type":"ContainerStarted","Data":"e1fb0c9cdd31a49d2bd48ad7ebe8bbe66fcd7d4129939d48a3e06fdeedec2d97"}
Nov 03 22:26:37.969252 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:37.969232    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:38.213772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:38.213726    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh" podStartSLOduration=2.026235479 podStartE2EDuration="7.213712705s" totalImagesPullingTime="4.708070451s" totalInitContainerRuntime="479.406775ms" isStatefulPod=false podCreationTimestamp="2026-11-03 22:26:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:38.16168725 +0000 UTC m=+201.426387294" watchObservedRunningTime="2026-11-03 22:26:38.213712705 +0000 UTC m=+201.478412715"
Nov 03 22:26:38.693781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:38.693745    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:40.083119 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:26:40.083081    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:26:40.985858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:40.985828    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s" event={"ID":"42413f4c-4836-4405-a2ce-b4b3d65f0ddf","Type":"ContainerStarted","Data":"77b205a73b6f594a5b76172ae48b0042315b5c127a32081bbeaade6cd78dbc56"}
Nov 03 22:26:41.106228 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.106177    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-phj7s" podStartSLOduration=2.03673089 podStartE2EDuration="5.106164692s" totalImagesPullingTime="3.069433802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:41.104745515 +0000 UTC m=+204.369445527" watchObservedRunningTime="2026-11-03 22:26:41.106164692 +0000 UTC m=+204.370864702"
Nov 03 22:26:41.833317 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.833279    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"]
Nov 03 22:26:41.839774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.839745    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:41.887790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.887765    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"]
Nov 03 22:26:41.932305 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.932276    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 03 22:26:41.937232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.937214    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-frlhn\""
Nov 03 22:26:41.937232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.937228    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:26:41.947319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.947299    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d658bfd8-7543-4369-a085-7a6ed61aed18-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh\" (UID: \"d658bfd8-7543-4369-a085-7a6ed61aed18\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:41.947386 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:41.947346    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jk6k\" (UniqueName: \"kubernetes.io/projected/d658bfd8-7543-4369-a085-7a6ed61aed18-kube-api-access-6jk6k\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh\" (UID: \"d658bfd8-7543-4369-a085-7a6ed61aed18\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:42.048439 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:42.048416    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d658bfd8-7543-4369-a085-7a6ed61aed18-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh\" (UID: \"d658bfd8-7543-4369-a085-7a6ed61aed18\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:42.048577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:42.048455    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jk6k\" (UniqueName: \"kubernetes.io/projected/d658bfd8-7543-4369-a085-7a6ed61aed18-kube-api-access-6jk6k\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh\" (UID: \"d658bfd8-7543-4369-a085-7a6ed61aed18\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:42.048784 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:42.048764    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d658bfd8-7543-4369-a085-7a6ed61aed18-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh\" (UID: \"d658bfd8-7543-4369-a085-7a6ed61aed18\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:42.111095 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:42.111029    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jk6k\" (UniqueName: \"kubernetes.io/projected/d658bfd8-7543-4369-a085-7a6ed61aed18-kube-api-access-6jk6k\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh\" (UID: \"d658bfd8-7543-4369-a085-7a6ed61aed18\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:42.150418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:42.150378    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"
Nov 03 22:26:42.615626 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:26:42.615594    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd658bfd8_7543_4369_a085_7a6ed61aed18.slice/crio-a6645fded6e9c520ebe9e2942f3a6b231ce955de2408c0536df8b57a4c1266f3 WatchSource:0}: Error finding container a6645fded6e9c520ebe9e2942f3a6b231ce955de2408c0536df8b57a4c1266f3: Status 404 returned error can't find the container with id a6645fded6e9c520ebe9e2942f3a6b231ce955de2408c0536df8b57a4c1266f3
Nov 03 22:26:42.617156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:42.617129    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh"]
Nov 03 22:26:42.997021 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:42.996986    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh" event={"ID":"d658bfd8-7543-4369-a085-7a6ed61aed18","Type":"ContainerStarted","Data":"a6645fded6e9c520ebe9e2942f3a6b231ce955de2408c0536df8b57a4c1266f3"}
Nov 03 22:26:43.412635 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:43.412526    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:44.734232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:44.734194    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:44.937850 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:44.937821    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:46.012035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:46.011994    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh" event={"ID":"d658bfd8-7543-4369-a085-7a6ed61aed18","Type":"ContainerStarted","Data":"33ea4f08e65ff4bc0d482b55c640bcde3246258e055c49a6fe7b1c07546c21c4"}
Nov 03 22:26:46.242210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:46.242152    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-8vrgh" podStartSLOduration=2.422346969 podStartE2EDuration="5.2421355s" totalImagesPullingTime="2.819788531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:46.240453762 +0000 UTC m=+209.505153773" watchObservedRunningTime="2026-11-03 22:26:46.2421355 +0000 UTC m=+209.506835512"
Nov 03 22:26:47.414815 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:47.414754    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:47.695323 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:47.695288    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:48.331616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:48.331576    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:48.976021 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:48.975994    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8bc866f6-qrjrh"
Nov 03 22:26:49.207136 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:49.207099    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:49.478870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:49.478829    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:50.075288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.075250    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:50.149733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.149703    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"]
Nov 03 22:26:50.153037 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.153021    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:50.210702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.210667    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 03 22:26:50.210816 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.210786    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 03 22:26:50.217180 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.217157    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"]
Nov 03 22:26:50.234401 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.234365    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wgpwz\""
Nov 03 22:26:50.321108 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.321082    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n8wp\" (UniqueName: \"kubernetes.io/projected/3d804340-862a-4e15-b483-972472da819b-kube-api-access-9n8wp\") pod \"cert-manager-cainjector-5c57bbd4fc-tbt26\" (UID: \"3d804340-862a-4e15-b483-972472da819b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:50.321214 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.321132    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d804340-862a-4e15-b483-972472da819b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-tbt26\" (UID: \"3d804340-862a-4e15-b483-972472da819b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:50.421971 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.421949    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n8wp\" (UniqueName: \"kubernetes.io/projected/3d804340-862a-4e15-b483-972472da819b-kube-api-access-9n8wp\") pod \"cert-manager-cainjector-5c57bbd4fc-tbt26\" (UID: \"3d804340-862a-4e15-b483-972472da819b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:50.422097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.422001    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d804340-862a-4e15-b483-972472da819b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-tbt26\" (UID: \"3d804340-862a-4e15-b483-972472da819b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:50.487366 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.487331    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d804340-862a-4e15-b483-972472da819b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-tbt26\" (UID: \"3d804340-862a-4e15-b483-972472da819b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:50.487484 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.487470    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n8wp\" (UniqueName: \"kubernetes.io/projected/3d804340-862a-4e15-b483-972472da819b-kube-api-access-9n8wp\") pod \"cert-manager-cainjector-5c57bbd4fc-tbt26\" (UID: \"3d804340-862a-4e15-b483-972472da819b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:50.764586 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:50.764494    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"
Nov 03 22:26:51.144997 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:26:51.144962    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d804340_862a_4e15_b483_972472da819b.slice/crio-ec4b547ae6fa4ba4c30ef56cd49ab60353c28f24628b17d9ba99d0d25a5918d1 WatchSource:0}: Error finding container ec4b547ae6fa4ba4c30ef56cd49ab60353c28f24628b17d9ba99d0d25a5918d1: Status 404 returned error can't find the container with id ec4b547ae6fa4ba4c30ef56cd49ab60353c28f24628b17d9ba99d0d25a5918d1
Nov 03 22:26:51.148058 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:51.148043    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26"]
Nov 03 22:26:52.040529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:52.040499    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26" event={"ID":"3d804340-862a-4e15-b483-972472da819b","Type":"ContainerStarted","Data":"ec4b547ae6fa4ba4c30ef56cd49ab60353c28f24628b17d9ba99d0d25a5918d1"}
Nov 03 22:26:54.133881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.133849    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:54.403999 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.403906    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:54.708635 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.708604    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"]
Nov 03 22:26:54.713102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.713082    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:54.774150 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.774123    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"]
Nov 03 22:26:54.824030 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.824005    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4zr2v\""
Nov 03 22:26:54.863992 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.863965    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbdc465a-e025-4548-82be-0992fbb8baf7-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-tqfqz\" (UID: \"cbdc465a-e025-4548-82be-0992fbb8baf7\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:54.864139 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.864010    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl56m\" (UniqueName: \"kubernetes.io/projected/cbdc465a-e025-4548-82be-0992fbb8baf7-kube-api-access-fl56m\") pod \"cert-manager-webhook-5dbbbf75cc-tqfqz\" (UID: \"cbdc465a-e025-4548-82be-0992fbb8baf7\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:54.965440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.965352    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl56m\" (UniqueName: \"kubernetes.io/projected/cbdc465a-e025-4548-82be-0992fbb8baf7-kube-api-access-fl56m\") pod \"cert-manager-webhook-5dbbbf75cc-tqfqz\" (UID: \"cbdc465a-e025-4548-82be-0992fbb8baf7\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:54.965576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:54.965502    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbdc465a-e025-4548-82be-0992fbb8baf7-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-tqfqz\" (UID: \"cbdc465a-e025-4548-82be-0992fbb8baf7\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:55.028522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:55.028482    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl56m\" (UniqueName: \"kubernetes.io/projected/cbdc465a-e025-4548-82be-0992fbb8baf7-kube-api-access-fl56m\") pod \"cert-manager-webhook-5dbbbf75cc-tqfqz\" (UID: \"cbdc465a-e025-4548-82be-0992fbb8baf7\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:55.038706 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:55.038683    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbdc465a-e025-4548-82be-0992fbb8baf7-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-tqfqz\" (UID: \"cbdc465a-e025-4548-82be-0992fbb8baf7\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:55.204679 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:55.204638    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:55.330459 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:55.330367    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:55.489079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:55.489042    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:55.709713 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:26:55.709686    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbdc465a_e025_4548_82be_0992fbb8baf7.slice/crio-6c5d53e777ae6707a2e4f498fe36b678b6245767cc7d08ece9feeb235218e80d WatchSource:0}: Error finding container 6c5d53e777ae6707a2e4f498fe36b678b6245767cc7d08ece9feeb235218e80d: Status 404 returned error can't find the container with id 6c5d53e777ae6707a2e4f498fe36b678b6245767cc7d08ece9feeb235218e80d
Nov 03 22:26:55.711356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:55.711335    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"]
Nov 03 22:26:55.714233 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:55.714186    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:56.061669 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:56.061590    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz" event={"ID":"cbdc465a-e025-4548-82be-0992fbb8baf7","Type":"ContainerStarted","Data":"6c5d53e777ae6707a2e4f498fe36b678b6245767cc7d08ece9feeb235218e80d"}
Nov 03 22:26:56.113333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:56.113296    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:57.640845 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:57.640812    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:26:58.072240 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:58.072198    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz" event={"ID":"cbdc465a-e025-4548-82be-0992fbb8baf7","Type":"ContainerStarted","Data":"5faaf384815ce7c74fd2f000907ce1047e3311fa7db873a4f156d52c0d5ca57b"}
Nov 03 22:26:58.072459 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:58.072361    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:26:58.073835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:58.073814    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26" event={"ID":"3d804340-862a-4e15-b483-972472da819b","Type":"ContainerStarted","Data":"1527cfb2b8f6ca8a4915aab9bc55007f3d919396e7fa9726ddf776309935143b"}
Nov 03 22:26:58.191678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:58.191623    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz" podStartSLOduration=2.451254881 podStartE2EDuration="4.191610297s" totalImagesPullingTime="1.740355416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:58.190839693 +0000 UTC m=+221.455539703" watchObservedRunningTime="2026-11-03 22:26:58.191610297 +0000 UTC m=+221.456310307"
Nov 03 22:26:59.644441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:26:59.644403    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:00.385990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:00.385956    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:00.714810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:00.714774    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:01.430276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:01.430245    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:01.643636 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:01.643604    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:01.900273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:01.900178    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:02.629475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:02.629440    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:03.282767 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.282730    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:03.310365 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.310313    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-tbt26" podStartSLOduration=7.373617136 podStartE2EDuration="13.310299601s" totalImagesPullingTime="5.936682465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:58.314490497 +0000 UTC m=+221.579190508" watchObservedRunningTime="2026-11-03 22:27:03.310299601 +0000 UTC m=+226.574999612"
Nov 03 22:27:03.310953 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.310936    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"]
Nov 03 22:27:03.314901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.314887    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.332731 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.332710    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377946e7-e808-40db-aebc-8714cd88b181-tls-certs\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.332847 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.332741    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w226\" (UniqueName: \"kubernetes.io/projected/377946e7-e808-40db-aebc-8714cd88b181-kube-api-access-7w226\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.332891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.332866    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/377946e7-e808-40db-aebc-8714cd88b181-tmp\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.422627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.422594    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"]
Nov 03 22:27:03.433264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.433240    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377946e7-e808-40db-aebc-8714cd88b181-tls-certs\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.433385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.433268    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w226\" (UniqueName: \"kubernetes.io/projected/377946e7-e808-40db-aebc-8714cd88b181-kube-api-access-7w226\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.433385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.433378    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/377946e7-e808-40db-aebc-8714cd88b181-tmp\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.435724 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.435687    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/377946e7-e808-40db-aebc-8714cd88b181-tmp\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.455668 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.455645    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 03 22:27:03.472629 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.472601    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 03 22:27:03.475889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.475867    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/377946e7-e808-40db-aebc-8714cd88b181-tls-certs\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.485262 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.485243    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j4bnj\""
Nov 03 22:27:03.499217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.499198    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w226\" (UniqueName: \"kubernetes.io/projected/377946e7-e808-40db-aebc-8714cd88b181-kube-api-access-7w226\") pod \"kube-auth-proxy-84cff947f5-5xh27\" (UID: \"377946e7-e808-40db-aebc-8714cd88b181\") " pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.560615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.560528    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:03.625861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.625830    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"
Nov 03 22:27:03.806082 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.806037    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"]
Nov 03 22:27:03.810435 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.810406    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:03.810753 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.810713    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836094    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836126    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836142    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836178    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836288    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836329    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836347    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8smh\" (UniqueName: \"kubernetes.io/projected/ce5c0c49-89bd-4f97-87ab-2d33724b913c-kube-api-access-b8smh\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836369    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.836723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.836435    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.865344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.865312    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jmp7x\""
Nov 03 22:27:03.874260 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.874225    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"]
Nov 03 22:27:03.937045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.936995    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937240 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937080    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937240 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937126    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937359 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937292    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8smh\" (UniqueName: \"kubernetes.io/projected/ce5c0c49-89bd-4f97-87ab-2d33724b913c-kube-api-access-b8smh\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937368    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937492 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937432    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937537 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937491    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937539    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937599    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937624    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.937923 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937895    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.938006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.937939    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.938237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.938215    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.938306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.938223    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.939892 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.939874    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:03.940086 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:03.940069    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:04.044946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.044917    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce5c0c49-89bd-4f97-87ab-2d33724b913c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:04.050299 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.050264    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8smh\" (UniqueName: \"kubernetes.io/projected/ce5c0c49-89bd-4f97-87ab-2d33724b913c-kube-api-access-b8smh\") pod \"data-science-gateway-data-science-gateway-class-76f56c5945rhvx4\" (UID: \"ce5c0c49-89bd-4f97-87ab-2d33724b913c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:04.074541 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:27:04.074517    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod377946e7_e808_40db_aebc_8714cd88b181.slice/crio-feaaea73267b82b022ed584be82357a5062646b1bbb4d7387640178026b990d6 WatchSource:0}: Error finding container feaaea73267b82b022ed584be82357a5062646b1bbb4d7387640178026b990d6: Status 404 returned error can't find the container with id feaaea73267b82b022ed584be82357a5062646b1bbb4d7387640178026b990d6
Nov 03 22:27:04.078021 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.078004    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84cff947f5-5xh27"]
Nov 03 22:27:04.080853 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.080834    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-tqfqz"
Nov 03 22:27:04.100998 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.100971    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27" event={"ID":"377946e7-e808-40db-aebc-8714cd88b181","Type":"ContainerStarted","Data":"feaaea73267b82b022ed584be82357a5062646b1bbb4d7387640178026b990d6"}
Nov 03 22:27:04.120124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.120097    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:04.124886 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.124862    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:04.344882 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.344790    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:04.516422 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:27:04.516373    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5c0c49_89bd_4f97_87ab_2d33724b913c.slice/crio-5972e3aca4edb5fec2129d1ae97790cadd75f082a2ee03161a7d4a67d750aaf1 WatchSource:0}: Error finding container 5972e3aca4edb5fec2129d1ae97790cadd75f082a2ee03161a7d4a67d750aaf1: Status 404 returned error can't find the container with id 5972e3aca4edb5fec2129d1ae97790cadd75f082a2ee03161a7d4a67d750aaf1
Nov 03 22:27:04.518443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.518422    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"]
Nov 03 22:27:04.518909 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.518877    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:27:04.518981 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.518936    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:27:04.518981 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.518973    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:27:04.559276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:04.559246    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:05.113130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:05.113093    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4" event={"ID":"ce5c0c49-89bd-4f97-87ab-2d33724b913c","Type":"ContainerStarted","Data":"65f5381973290ac113158521e0a9fad17dfcc9b13e141072a048910e8ab4ffca"}
Nov 03 22:27:05.113301 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:05.113183    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4" event={"ID":"ce5c0c49-89bd-4f97-87ab-2d33724b913c","Type":"ContainerStarted","Data":"5972e3aca4edb5fec2129d1ae97790cadd75f082a2ee03161a7d4a67d750aaf1"}
Nov 03 22:27:05.125073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:05.125047    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:05.129336 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:05.129316    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:05.238961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:05.238915    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4" podStartSLOduration=2.238901759 podStartE2EDuration="2.238901759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:27:05.237906134 +0000 UTC m=+228.502606144" watchObservedRunningTime="2026-11-03 22:27:05.238901759 +0000 UTC m=+228.503601769"
Nov 03 22:27:06.118405 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:06.118367    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:06.119434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:06.119411    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4"
Nov 03 22:27:08.129809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:08.129767    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27" event={"ID":"377946e7-e808-40db-aebc-8714cd88b181","Type":"ContainerStarted","Data":"ccea8c0e1f02f7fe9a5c5e80bf2afee2a589003c53597f077654236f871ff954"}
Nov 03 22:27:08.253957 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:08.253910    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84cff947f5-5xh27" podStartSLOduration=1.937262004 podStartE2EDuration="5.253895767s" totalImagesPullingTime="3.316633763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:27:08.253438856 +0000 UTC m=+231.518138861" watchObservedRunningTime="2026-11-03 22:27:08.253895767 +0000 UTC m=+231.518595777"
Nov 03 22:27:09.108628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:09.108590    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:09.363439 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:09.363331    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:09.565110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:09.565079    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:09.903904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:09.903861    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:10.083838 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:27:10.083799    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:27:10.105496 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:10.105463    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:10.831780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:10.831749    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:15.222732 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:15.222698    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:15.433379 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:15.433343    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:16.553768 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.553736    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"]
Nov 03 22:27:16.557131 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.557114    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.635927 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.635897    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc6acd5-eaae-433a-b24f-6289586ba946-metrics-certs\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.636106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.635962    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7cc6acd5-eaae-433a-b24f-6289586ba946-manager-config\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.636106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.636008    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvpt9\" (UniqueName: \"kubernetes.io/projected/7cc6acd5-eaae-433a-b24f-6289586ba946-kube-api-access-nvpt9\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.636197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.636161    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7cc6acd5-eaae-433a-b24f-6289586ba946-cert\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.669934 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.669911    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 03 22:27:16.670284 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.670261    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-k69fc\""
Nov 03 22:27:16.670284 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.670284    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 03 22:27:16.670471 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.670338    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 03 22:27:16.671917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.671892    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"]
Nov 03 22:27:16.737520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.737488    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7cc6acd5-eaae-433a-b24f-6289586ba946-manager-config\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.737520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.737520    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvpt9\" (UniqueName: \"kubernetes.io/projected/7cc6acd5-eaae-433a-b24f-6289586ba946-kube-api-access-nvpt9\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.737757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.737577    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7cc6acd5-eaae-433a-b24f-6289586ba946-cert\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.737757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.737603    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc6acd5-eaae-433a-b24f-6289586ba946-metrics-certs\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.738163 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.738144    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7cc6acd5-eaae-433a-b24f-6289586ba946-manager-config\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.740022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.740001    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc6acd5-eaae-433a-b24f-6289586ba946-metrics-certs\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.740116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.740060    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7cc6acd5-eaae-433a-b24f-6289586ba946-cert\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.802168 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.802136    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvpt9\" (UniqueName: \"kubernetes.io/projected/7cc6acd5-eaae-433a-b24f-6289586ba946-kube-api-access-nvpt9\") pod \"jobset-controller-manager-6bf7f5894-7chcw\" (UID: \"7cc6acd5-eaae-433a-b24f-6289586ba946\") " pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:16.922355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.922336    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-k69fc\""
Nov 03 22:27:16.929756 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:16.929736    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:17.299709 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:27:17.299677    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc6acd5_eaae_433a_b24f_6289586ba946.slice/crio-3385d44aae91dfd54a091bb19420d955c8665f98875e6c8c715f6a1ec85959d3 WatchSource:0}: Error finding container 3385d44aae91dfd54a091bb19420d955c8665f98875e6c8c715f6a1ec85959d3: Status 404 returned error can't find the container with id 3385d44aae91dfd54a091bb19420d955c8665f98875e6c8c715f6a1ec85959d3
Nov 03 22:27:17.301311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:17.301284    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"]
Nov 03 22:27:17.719920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:17.719876    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:17.939040 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:17.939006    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:18.175877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:18.175846    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw" event={"ID":"7cc6acd5-eaae-433a-b24f-6289586ba946","Type":"ContainerStarted","Data":"3385d44aae91dfd54a091bb19420d955c8665f98875e6c8c715f6a1ec85959d3"}
Nov 03 22:27:20.188751 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:20.188715    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:20.400449 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:20.400380    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:21.596583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:21.596556    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:21.798245 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:21.798160    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:22.196987 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:22.196954    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw" event={"ID":"7cc6acd5-eaae-433a-b24f-6289586ba946","Type":"ContainerStarted","Data":"f8ea6c24b812eda40863b154d034c353bdbea668825149ef44341a80b5579146"}
Nov 03 22:27:22.197163 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:22.197066    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:22.316611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:22.316560    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw" podStartSLOduration=2.069667005 podStartE2EDuration="6.316546239s" totalImagesPullingTime="4.246879234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:27:22.315800313 +0000 UTC m=+245.580500323" watchObservedRunningTime="2026-11-03 22:27:22.316546239 +0000 UTC m=+245.581246249"
Nov 03 22:27:24.920578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:24.920532    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:25.169859 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:25.169825    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:29.098096 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:29.098046    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:30.149154 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:30.149118    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:30.408603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:30.408529    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:30.611652 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:30.611620    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:30.835985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:30.835942    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:33.207842 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:33.207814    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-6bf7f5894-7chcw"
Nov 03 22:27:33.765755 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:33.765719    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:39.165601 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:39.165567    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:39.477305 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:39.477273    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:40.082872 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:27:40.082836    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:27:41.622971 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:41.622916    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:41.878465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:41.878364    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:45.894929 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:45.894894    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:46.114803 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:46.114775    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:46.397513 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:46.397476    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:46.604027 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:46.603996    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:52.632258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:52.632226    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:52.923893 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:52.923864    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:55.667407 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:55.667357    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:27:55.866455 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:27:55.866416    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:10.082505 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:28:10.082428    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:28:10.344459 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:10.344349    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:10.610654 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:10.610562    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:11.148402 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:11.148359    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:32.929469 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:32.929433    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:33.134490 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:33.134453    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:33.340217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:33.340135    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:34.157383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:34.157345    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:36.079665 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:36.079632    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:36.279366 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:36.279330    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:36.476582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:36.476546    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:36.679321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:36.679287    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:36.885663 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:36.885575    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:37.089979 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:37.089923    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:37.994661 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:37.994612    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:38.244231 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:38.244192    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:38.472856 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:38.472801    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:40.082727 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:28:40.082695    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:28:41.476972 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.476939    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"]
Nov 03 22:28:41.480570 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.480546    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.537156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.537074    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 03 22:28:41.537156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.537075    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 03 22:28:41.537156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.537119    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 03 22:28:41.537156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.537082    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 03 22:28:41.588620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.588593    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-f2zw5\""
Nov 03 22:28:41.588898 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.588872    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 03 22:28:41.590962 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.590934    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"]
Nov 03 22:28:41.600171 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.600146    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.600290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.600195    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sszjl\" (UniqueName: \"kubernetes.io/projected/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-kube-api-access-sszjl\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.600290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.600234    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.600583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.600451    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.690617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.690573    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:41.701587 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.701561    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.701726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.701600    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszjl\" (UniqueName: \"kubernetes.io/projected/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-kube-api-access-sszjl\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.701726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.701622    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.701726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.701697    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.701892 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:28:41.701763    5162 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 03 22:28:41.701892 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:28:41.701844    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-proxy-source podName:c22f1c68-cb3b-49d3-bda7-b70a4aba7265 nodeName:}" failed. No retries permitted until 2026-11-03 22:28:42.201822496 +0000 UTC m=+325.466522504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-hqw4s" (UID: "c22f1c68-cb3b-49d3-bda7-b70a4aba7265") : configmap "usage-tenancy-proxy-source" not found
Nov 03 22:28:41.702311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.702291    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.704108 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.704090    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.766307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.766246    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sszjl\" (UniqueName: \"kubernetes.io/projected/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-kube-api-access-sszjl\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:41.938320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:41.938287    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:42.207765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:42.207732    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:42.208578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:42.208553    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/c22f1c68-cb3b-49d3-bda7-b70a4aba7265-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-hqw4s\" (UID: \"c22f1c68-cb3b-49d3-bda7-b70a4aba7265\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:42.393095 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:42.393055    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:42.763125 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:28:42.763093    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22f1c68_cb3b_49d3_bda7_b70a4aba7265.slice/crio-84f96fb528bec6018eee3ef8b5ec180f39e9546e468592f82dd6495783779c12 WatchSource:0}: Error finding container 84f96fb528bec6018eee3ef8b5ec180f39e9546e468592f82dd6495783779c12: Status 404 returned error can't find the container with id 84f96fb528bec6018eee3ef8b5ec180f39e9546e468592f82dd6495783779c12
Nov 03 22:28:42.767227 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:42.767193    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"]
Nov 03 22:28:42.769939 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:42.769914    5162 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:28:43.576999 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:43.576963    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s" event={"ID":"c22f1c68-cb3b-49d3-bda7-b70a4aba7265","Type":"ContainerStarted","Data":"84f96fb528bec6018eee3ef8b5ec180f39e9546e468592f82dd6495783779c12"}
Nov 03 22:28:43.681929 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:43.681893    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:43.891876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:43.891791    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:44.091858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:44.091819    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:44.289310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:44.289278    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:44.490654 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:44.490616    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:44.689932 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:44.689903    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:44.947382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:44.947298    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:55.648540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:55.648503    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s" event={"ID":"c22f1c68-cb3b-49d3-bda7-b70a4aba7265","Type":"ContainerStarted","Data":"f66f6ecd8fae12ed4d4edb1bcc716239931fda89c6741d78a051c630701fb9e1"}
Nov 03 22:28:55.648898 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:55.648577    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:28:55.769088 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:55.768998    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s" podStartSLOduration=2.040430353 podStartE2EDuration="14.768985314s" totalImagesPullingTime="12.728554961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:28:55.768447368 +0000 UTC m=+339.033147379" watchObservedRunningTime="2026-11-03 22:28:55.768985314 +0000 UTC m=+339.033685357"
Nov 03 22:28:57.461613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:57.461574    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:28:57.665602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:28:57.665570    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:06.658811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:06.658770    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-hqw4s"
Nov 03 22:29:10.083008 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:29:10.082970    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:29:11.730428 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:11.730366    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:11.936208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:11.936169    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:13.847673 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:13.847640    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:14.068754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:14.068721    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:14.266277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:14.266245    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:24.483569 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:24.483534    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:24.701502 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:24.701464    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:24.906512 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:24.906433    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"]
Nov 03 22:29:24.938897 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:24.938872    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:24.969295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:24.969270    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"]
Nov 03 22:29:25.018647 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.018618    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 22:29:25.045205 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.045182    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 22:29:25.045205 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.045200    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9qhjs\""
Nov 03 22:29:25.045341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.045216    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 22:29:25.045341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.045233    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 03 22:29:25.094257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.094231    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-metrics-certs\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.094418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.094271    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-879kj\" (UniqueName: \"kubernetes.io/projected/44df9647-34b3-425f-b3f8-82719b9b14e0-kube-api-access-879kj\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.094500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.094442    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-maas-api-tls\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.094548 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.094495    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44df9647-34b3-425f-b3f8-82719b9b14e0-openshift-service-ca\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.195966 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.195936    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44df9647-34b3-425f-b3f8-82719b9b14e0-openshift-service-ca\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.196109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.195979    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-metrics-certs\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.196109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.196006    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-879kj\" (UniqueName: \"kubernetes.io/projected/44df9647-34b3-425f-b3f8-82719b9b14e0-kube-api-access-879kj\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.196109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.196067    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-maas-api-tls\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.196664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.196639    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44df9647-34b3-425f-b3f8-82719b9b14e0-openshift-service-ca\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.198416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.198378    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-metrics-certs\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.198487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.198460    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-maas-api-tls\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.258600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.258575    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-879kj\" (UniqueName: \"kubernetes.io/projected/44df9647-34b3-425f-b3f8-82719b9b14e0-kube-api-access-879kj\") pod \"maas-api-797d89b4d5-mx798\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") " pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.550254 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.550174    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:25.918929 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:29:25.918900    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44df9647_34b3_425f_b3f8_82719b9b14e0.slice/crio-62c0047bfff97f1023e34c1f7ef568dc074cf29c585d7187b0ef2d6e580f940d WatchSource:0}: Error finding container 62c0047bfff97f1023e34c1f7ef568dc074cf29c585d7187b0ef2d6e580f940d: Status 404 returned error can't find the container with id 62c0047bfff97f1023e34c1f7ef568dc074cf29c585d7187b0ef2d6e580f940d
Nov 03 22:29:25.921038 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:25.921014    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"]
Nov 03 22:29:26.812330 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:26.812296    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798" event={"ID":"44df9647-34b3-425f-b3f8-82719b9b14e0","Type":"ContainerStarted","Data":"62c0047bfff97f1023e34c1f7ef568dc074cf29c585d7187b0ef2d6e580f940d"}
Nov 03 22:29:29.828949 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:29.828910    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798" event={"ID":"44df9647-34b3-425f-b3f8-82719b9b14e0","Type":"ContainerStarted","Data":"3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152"}
Nov 03 22:29:29.829326 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:29.829066    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:29.951639 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:29.951587    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798" podStartSLOduration=2.8604111 podStartE2EDuration="5.951573187s" totalImagesPullingTime="3.091162087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:29:29.95001039 +0000 UTC m=+373.214710405" watchObservedRunningTime="2026-11-03 22:29:29.951573187 +0000 UTC m=+373.216273242"
Nov 03 22:29:34.455472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:34.455435    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:35.839588 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:35.839553    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 22:29:40.083779 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:29:40.083700    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:29:44.484913 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:44.484881    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:48.704418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:48.704372    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:29:48.950362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:29:48.950329    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:10.081980 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:30:10.081946    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:30:23.084043 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:23.084011    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:23.338629 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:23.338553    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:34.802182 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:34.802142    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:40.083111 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:30:40.083076    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:30:43.022697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:43.022662    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:43.809527 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:43.809491    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:44.014019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:44.013981    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:44.818261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:44.818228    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:46.268597 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:46.268561    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:49.147597 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.147567    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"]
Nov 03 22:30:49.150760 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.150744    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.227492 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.227459    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.227638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.227520    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.227638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.227587    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx5k6\" (UniqueName: \"kubernetes.io/projected/4def062f-125a-468b-a494-9aa4d4f99ef7-kube-api-access-nx5k6\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.227638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.227619    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.227638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.227636    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.227850 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.227777    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4def062f-125a-468b-a494-9aa4d4f99ef7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.256325 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.256291    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 03 22:30:49.256325 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.256306    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 03 22:30:49.256519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.256327    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pssl4\""
Nov 03 22:30:49.256557 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.256543    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 03 22:30:49.258579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.258562    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"]
Nov 03 22:30:49.291619 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.291592    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:49.329052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329028    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329059    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx5k6\" (UniqueName: \"kubernetes.io/projected/4def062f-125a-468b-a494-9aa4d4f99ef7-kube-api-access-nx5k6\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329079    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329094    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329223    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4def062f-125a-468b-a494-9aa4d4f99ef7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329313    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329476    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329578    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.329676 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.329658    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.331446 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.331421    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4def062f-125a-468b-a494-9aa4d4f99ef7-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.331693 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.331666    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4def062f-125a-468b-a494-9aa4d4f99ef7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.391987 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.391957    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx5k6\" (UniqueName: \"kubernetes.io/projected/4def062f-125a-468b-a494-9aa4d4f99ef7-kube-api-access-nx5k6\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2\" (UID: \"4def062f-125a-468b-a494-9aa4d4f99ef7\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.462869 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.462840    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:49.832779 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:30:49.832747    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4def062f_125a_468b_a494_9aa4d4f99ef7.slice/crio-d247a0b7352900b9e9ba8bb69a2e109f184eb3fe15dd1d2dfeaaf4cbca798e0b WatchSource:0}: Error finding container d247a0b7352900b9e9ba8bb69a2e109f184eb3fe15dd1d2dfeaaf4cbca798e0b: Status 404 returned error can't find the container with id d247a0b7352900b9e9ba8bb69a2e109f184eb3fe15dd1d2dfeaaf4cbca798e0b
Nov 03 22:30:49.834674 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:49.834655    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"]
Nov 03 22:30:50.207019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:50.206985    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2" event={"ID":"4def062f-125a-468b-a494-9aa4d4f99ef7","Type":"ContainerStarted","Data":"d247a0b7352900b9e9ba8bb69a2e109f184eb3fe15dd1d2dfeaaf4cbca798e0b"}
Nov 03 22:30:52.219183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.219146    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2" event={"ID":"4def062f-125a-468b-a494-9aa4d4f99ef7","Type":"ContainerStarted","Data":"ba485be761aeb803dbb5463ad5c533b618f2dd393fc7677121b3d45d989526d3"}
Nov 03 22:30:52.328319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.328290    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"]
Nov 03 22:30:52.331888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.331872    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.342629 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.342608    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"]
Nov 03 22:30:52.379100 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.379039    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2" podStartSLOduration=1.8841922869999999 podStartE2EDuration="3.379026229s" totalImagesPullingTime="1.494833942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:30:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:30:52.377571808 +0000 UTC m=+455.642271818" watchObservedRunningTime="2026-11-03 22:30:52.379026229 +0000 UTC m=+455.643726217"
Nov 03 22:30:52.391456 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.391432    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 03 22:30:52.429879 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.429852    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:52.457982 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.457956    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8vxx\" (UniqueName: \"kubernetes.io/projected/9a63990d-9392-4c4a-b804-3dfa65e4a824-kube-api-access-d8vxx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.458094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.457994    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.458094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.458055    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.458094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.458078    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.458273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.458115    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.458273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.458185    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a63990d-9392-4c4a-b804-3dfa65e4a824-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.463820 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.463804    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:52.558759 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.558695    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.558759 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.558733    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.558902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.558783    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.558902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.558818    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a63990d-9392-4c4a-b804-3dfa65e4a824-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.558902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.558872    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8vxx\" (UniqueName: \"kubernetes.io/projected/9a63990d-9392-4c4a-b804-3dfa65e4a824-kube-api-access-d8vxx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.559025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.558913    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.559137 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.559119    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.559215 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.559192    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.559269 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.559230    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.561042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.561015    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a63990d-9392-4c4a-b804-3dfa65e4a824-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.561294 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.561276    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a63990d-9392-4c4a-b804-3dfa65e4a824-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.621408 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.621372    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8vxx\" (UniqueName: \"kubernetes.io/projected/9a63990d-9392-4c4a-b804-3dfa65e4a824-kube-api-access-d8vxx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp\" (UID: \"9a63990d-9392-4c4a-b804-3dfa65e4a824\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:52.644108 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:52.644085    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:53.012899 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:30:53.012867    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a63990d_9392_4c4a_b804_3dfa65e4a824.slice/crio-f08aeef9ffbf4c1b1e9566062c2869f8697088308cf7796d83516bfe9b10caf0 WatchSource:0}: Error finding container f08aeef9ffbf4c1b1e9566062c2869f8697088308cf7796d83516bfe9b10caf0: Status 404 returned error can't find the container with id f08aeef9ffbf4c1b1e9566062c2869f8697088308cf7796d83516bfe9b10caf0
Nov 03 22:30:53.015372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:53.015351    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"]
Nov 03 22:30:53.226704 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:53.226669    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" event={"ID":"9a63990d-9392-4c4a-b804-3dfa65e4a824","Type":"ContainerStarted","Data":"f08aeef9ffbf4c1b1e9566062c2869f8697088308cf7796d83516bfe9b10caf0"}
Nov 03 22:30:54.232887 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:54.232839    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" event={"ID":"9a63990d-9392-4c4a-b804-3dfa65e4a824","Type":"ContainerStarted","Data":"7f3f826bda97487c57bf1112dbabc1622fb3bc45b35a75a04e02029cd7204af0"}
Nov 03 22:30:54.389739 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:54.389686    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podStartSLOduration=1.776411119 podStartE2EDuration="2.389672731s" totalImagesPullingTime="613.261612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:30:54.387884085 +0000 UTC m=+457.652584098" watchObservedRunningTime="2026-11-03 22:30:54.389672731 +0000 UTC m=+457.654372738"
Nov 03 22:30:54.644726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:54.644653    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:30:55.191622 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:55.191578    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:59.232597 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.232563    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"]
Nov 03 22:30:59.236140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.236124    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.318317 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.318283    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.318521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.318331    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.318521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.318346    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.318521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.318505    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.318692 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.318576    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkdfk\" (UniqueName: \"kubernetes.io/projected/26e27a85-625a-45c5-aba9-d60be04671b7-kube-api-access-jkdfk\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.318692 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.318595    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e27a85-625a-45c5-aba9-d60be04671b7-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.344197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.344167    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 03 22:30:59.346304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.346283    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"]
Nov 03 22:30:59.381385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.381353    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:30:59.419420 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419376    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkdfk\" (UniqueName: \"kubernetes.io/projected/26e27a85-625a-45c5-aba9-d60be04671b7-kube-api-access-jkdfk\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.419512 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419428    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e27a85-625a-45c5-aba9-d60be04671b7-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.419512 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419459    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.419512 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419489    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.419512 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419505    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.419711 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419552    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.419889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419861    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.420015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419923    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.420015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.419963    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.421752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.421731    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e27a85-625a-45c5-aba9-d60be04671b7-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.421984 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.421967    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e27a85-625a-45c5-aba9-d60be04671b7-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.464011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.463989    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:59.476508 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.476492    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:30:59.481658 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.481634    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkdfk\" (UniqueName: \"kubernetes.io/projected/26e27a85-625a-45c5-aba9-d60be04671b7-kube-api-access-jkdfk\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4lpz9\" (UID: \"26e27a85-625a-45c5-aba9-d60be04671b7\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.547959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.547887    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:30:59.920477 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:30:59.920447    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e27a85_625a_45c5_aba9_d60be04671b7.slice/crio-fb027f1ce5ed441feb5954f19465910ab17e4ab456a76321bbb25d4f5abb25b9 WatchSource:0}: Error finding container fb027f1ce5ed441feb5954f19465910ab17e4ab456a76321bbb25d4f5abb25b9: Status 404 returned error can't find the container with id fb027f1ce5ed441feb5954f19465910ab17e4ab456a76321bbb25d4f5abb25b9
Nov 03 22:30:59.922434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:30:59.922317    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"]
Nov 03 22:31:00.263716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:00.263678    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9" event={"ID":"26e27a85-625a-45c5-aba9-d60be04671b7","Type":"ContainerStarted","Data":"fb027f1ce5ed441feb5954f19465910ab17e4ab456a76321bbb25d4f5abb25b9"}
Nov 03 22:31:00.274903 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:00.274879    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2"
Nov 03 22:31:01.270725 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:01.270683    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9" event={"ID":"26e27a85-625a-45c5-aba9-d60be04671b7","Type":"ContainerStarted","Data":"b322deb9acdcec8dbb2c5682c9626235ecd083b6c59cbf43fdb1b996c461b696"}
Nov 03 22:31:01.393179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:01.393128    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9" podStartSLOduration=1.5283494640000002 podStartE2EDuration="2.39311456s" totalImagesPullingTime="864.765096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:31:01.391356388 +0000 UTC m=+464.656056398" watchObservedRunningTime="2026-11-03 22:31:01.39311456 +0000 UTC m=+464.657814597"
Nov 03 22:31:01.548421 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:01.548329    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:31:02.618937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:02.618904    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:02.644724 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:02.644697    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:31:02.657180 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:02.657157    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:31:03.291195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:03.291166    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:31:06.201300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.201261    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:06.355121 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.355090    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"]
Nov 03 22:31:06.360355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.360339    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.417756 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.417729    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"]
Nov 03 22:31:06.466873 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.466803    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 03 22:31:06.486484 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.486453    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.486623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.486542    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nz8t\" (UniqueName: \"kubernetes.io/projected/c4cff4d6-fdcd-4171-956e-046ae097bca5-kube-api-access-6nz8t\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.486623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.486615    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.486787 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.486751    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.486944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.486836    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.486944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.486905    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cff4d6-fdcd-4171-956e-046ae097bca5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.494835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.494812    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:06.587449 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.587422    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.587579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.587474    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nz8t\" (UniqueName: \"kubernetes.io/projected/c4cff4d6-fdcd-4171-956e-046ae097bca5-kube-api-access-6nz8t\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.587579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.587545    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.587664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.587594    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.587664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.587634    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.587730 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.587696    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cff4d6-fdcd-4171-956e-046ae097bca5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.587970 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.587951    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.588052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.588031    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.588102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.588066    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.589868 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.589849    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cff4d6-fdcd-4171-956e-046ae097bca5-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.590159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.590140    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cff4d6-fdcd-4171-956e-046ae097bca5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.650320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.650288    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nz8t\" (UniqueName: \"kubernetes.io/projected/c4cff4d6-fdcd-4171-956e-046ae097bca5-kube-api-access-6nz8t\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-bjwpl\" (UID: \"c4cff4d6-fdcd-4171-956e-046ae097bca5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:06.673171 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:06.673146    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:07.042688 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:31:07.042654    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cff4d6_fdcd_4171_956e_046ae097bca5.slice/crio-450569a09096e9b6aca445d149fb33a991c5f7972a160b6b1760c60b4499dcb3 WatchSource:0}: Error finding container 450569a09096e9b6aca445d149fb33a991c5f7972a160b6b1760c60b4499dcb3: Status 404 returned error can't find the container with id 450569a09096e9b6aca445d149fb33a991c5f7972a160b6b1760c60b4499dcb3
Nov 03 22:31:07.045353 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:07.045296    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"]
Nov 03 22:31:07.301816 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:07.301714    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl" event={"ID":"c4cff4d6-fdcd-4171-956e-046ae097bca5","Type":"ContainerStarted","Data":"450569a09096e9b6aca445d149fb33a991c5f7972a160b6b1760c60b4499dcb3"}
Nov 03 22:31:08.308458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:08.308418    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl" event={"ID":"c4cff4d6-fdcd-4171-956e-046ae097bca5","Type":"ContainerStarted","Data":"13c6b5d5347ef88a7d3927f0bdba1cc2faaff1d4379f434852be06f299342234"}
Nov 03 22:31:08.431064 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:08.431011    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl" podStartSLOduration=1.6883428409999999 podStartE2EDuration="2.430996666s" totalImagesPullingTime="742.653825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:31:08.430319617 +0000 UTC m=+471.695019628" watchObservedRunningTime="2026-11-03 22:31:08.430996666 +0000 UTC m=+471.695696677"
Nov 03 22:31:08.673536 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:08.673504    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:09.548689 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:09.548654    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:31:09.560550 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:09.560525    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:31:10.083460 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:31:10.083426    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:31:10.329600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:10.329569    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4lpz9"
Nov 03 22:31:16.051628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.051598    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"]
Nov 03 22:31:16.055919 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.055903    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068342    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068491 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068368    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068491 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068414    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068525    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068565    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068677    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068750 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068708    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068750 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068732    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84zvk\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-kube-api-access-84zvk\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.068830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.068823    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.103317 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.103292    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"]
Nov 03 22:31:16.107012 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.106998    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.116263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.116244    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v5vrs\""
Nov 03 22:31:16.158561 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.158536    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"]
Nov 03 22:31:16.169167 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169144    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169188    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.169291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169210    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169233    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169283    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169570 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169304    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169570 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169490    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169679 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169575    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.169679 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169605    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.169790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169677    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6v97\" (UniqueName: \"kubernetes.io/projected/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-kube-api-access-r6v97\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.169790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169723    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169757    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.169790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169764    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169816    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169826    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169853    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169925    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84zvk\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-kube-api-access-84zvk\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.169983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169969    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.170222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.169998    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.170222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.170021    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.170222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.170038    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.170222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.170172    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.170222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.170215    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.171974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.171948    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.172152 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.172135    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.209861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.209832    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"]
Nov 03 22:31:16.232916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.232878    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84zvk\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-kube-api-access-84zvk\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.233434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.233413    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-q2jqm\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.270974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.270939    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271156    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271305 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271199    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6v97\" (UniqueName: \"kubernetes.io/projected/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-kube-api-access-r6v97\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271305 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271266    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271305 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271296    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271497 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271318    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271497 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271365    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271497 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271407    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271497 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271436    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271506    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.271858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.271832    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.272191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.272168    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.272288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.272201    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.272347 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.272301    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.273887 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.273867    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.274536 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.274510    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.333661 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.333602    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.333758 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.333742    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6v97\" (UniqueName: \"kubernetes.io/projected/e2eb0dd5-b5d0-40b6-a00b-7761d4adadda-kube-api-access-r6v97\") pod \"maas-default-gateway-openshift-default-59f957fdd7-p59kr\" (UID: \"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.368916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.368890    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:16.420290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.419481    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:16.673607 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.673583    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:16.686982 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.686956    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:16.789143 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:31:16.789116    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b3b9d5_4ba8_44dd_9e78_b75025735de8.slice/crio-08f340a29a3761ee67c203647ac767aac818c031dbde773bb7f4d107cc444c4b WatchSource:0}: Error finding container 08f340a29a3761ee67c203647ac767aac818c031dbde773bb7f4d107cc444c4b: Status 404 returned error can't find the container with id 08f340a29a3761ee67c203647ac767aac818c031dbde773bb7f4d107cc444c4b
Nov 03 22:31:16.791446 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.791414    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:31:16.791561 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.791475    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:31:16.791561 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.791507    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:31:16.792043 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.792021    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"]
Nov 03 22:31:16.839195 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:31:16.839168    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2eb0dd5_b5d0_40b6_a00b_7761d4adadda.slice/crio-f63923eb1f779d35255aafa8a3d00a3c0690aeb543e3cda98c2e145f0c35d285 WatchSource:0}: Error finding container f63923eb1f779d35255aafa8a3d00a3c0690aeb543e3cda98c2e145f0c35d285: Status 404 returned error can't find the container with id f63923eb1f779d35255aafa8a3d00a3c0690aeb543e3cda98c2e145f0c35d285
Nov 03 22:31:16.841239 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.841211    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:31:16.841298 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.841271    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:31:16.841338 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.841296    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:31:16.842535 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:16.842510    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"]
Nov 03 22:31:17.371073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.371027    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" event={"ID":"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda","Type":"ContainerStarted","Data":"1f7f74afe060fa10979ca1cec71209ca97384b9636dfdb114de8657baebb0149"}
Nov 03 22:31:17.375864 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.375831    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" event={"ID":"e2eb0dd5-b5d0-40b6-a00b-7761d4adadda","Type":"ContainerStarted","Data":"f63923eb1f779d35255aafa8a3d00a3c0690aeb543e3cda98c2e145f0c35d285"}
Nov 03 22:31:17.384660 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.384627    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" event={"ID":"02b3b9d5-4ba8-44dd-9e78-b75025735de8","Type":"ContainerStarted","Data":"b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761"}
Nov 03 22:31:17.384837 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.384679    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" event={"ID":"02b3b9d5-4ba8-44dd-9e78-b75025735de8","Type":"ContainerStarted","Data":"08f340a29a3761ee67c203647ac767aac818c031dbde773bb7f4d107cc444c4b"}
Nov 03 22:31:17.404437 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.403529    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-bjwpl"
Nov 03 22:31:17.419974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.419952    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:17.421551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.421519    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:31:17.421650 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.421581    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:31:17.505060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.504999    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podStartSLOduration=1.50498219 podStartE2EDuration="1.50498219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:31:17.502208005 +0000 UTC m=+480.766908014" watchObservedRunningTime="2026-11-03 22:31:17.50498219 +0000 UTC m=+480.769682204"
Nov 03 22:31:17.791480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:17.781597    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podStartSLOduration=1.781575017 podStartE2EDuration="1.781575017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:31:17.778913162 +0000 UTC m=+481.043613174" watchObservedRunningTime="2026-11-03 22:31:17.781575017 +0000 UTC m=+481.046275029"
Nov 03 22:31:18.369836 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:18.369796    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:18.374542 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:18.374509    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:31:18.375089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:18.375056    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:31:18.420285 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:18.420252    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:31:18.420455 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:18.420296    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:31:19.377415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:19.374495    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:31:19.377415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:19.374556    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:31:19.419954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:19.419904    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:31:19.420110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:19.419974    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:31:20.378415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:20.377774    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:31:20.378415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:20.377835    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:31:20.420409 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:20.420367    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:31:20.420580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:20.420431    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:31:21.374413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:21.373739    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:31:21.374413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:21.373836    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:31:21.420319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:21.420286    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:31:21.420694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:21.420338    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:31:22.385417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:22.380766    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:31:22.385417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:22.380828    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:31:22.420364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:22.420333    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:31:22.420754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:22.420377    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:31:23.369306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:23.369271    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:31:23.369515 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:23.369326    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:31:23.420398 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:23.420354    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:31:23.420773 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:23.420418    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:31:23.650429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:23.650319    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:23.884622 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:23.884583    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:25.138937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:25.138902    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:25.139841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:25.139820    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:25.140520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:25.140497    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 22:31:25.154498 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:25.154478    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:25.154788 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:25.154765    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:25.155631 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:25.155614    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr"
Nov 03 22:31:28.221645 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:28.221611    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:32.568683 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:32.568648    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:32.826069 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:32.825974    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:35.155356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:35.155320    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:40.084268 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:31:40.084235    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:31:45.182507 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:45.182469    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:57.359675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:57.359638    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:57.874274 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:57.874240    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:31:59.503608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:31:59.503559    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:01.763783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:01.763748    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:09.590659 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:09.590617    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:09.793977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:09.793941    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:10.083385 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:32:10.083351    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:32:35.587106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:35.587068    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:40.082714 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:32:40.082641    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:32:43.163709 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.163672    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:43.421501 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.421463    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:43.645592 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.645556    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"]
Nov 03 22:32:43.649130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.649109    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.704786 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.704701    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-961054-openshift-default-dockercfg-kjq9m\""
Nov 03 22:32:43.763971 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.763933    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.763971 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.763974    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-envoy\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.764177 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.764000    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8d4h\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-kube-api-access-f8d4h\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.764177 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.764067    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-data\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.764177 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.764157    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-certs\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.764177 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.764176    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-podinfo\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.764304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.764221    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91745d21-1539-4c6a-a38d-250b69eb48ef-istiod-ca-cert\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.764340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.764295    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-credential-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.764376 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.764361    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-token\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.782010 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.781981    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:43.865780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.865756    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8d4h\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-kube-api-access-f8d4h\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.865933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.865795    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-data\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.865933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.865846    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-certs\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.865933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.865874    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-podinfo\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866114 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.865934    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91745d21-1539-4c6a-a38d-250b69eb48ef-istiod-ca-cert\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866114 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.865969    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-credential-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866114 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866033    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-token\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866114 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866094    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866322 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866129    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-envoy\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866322 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866310    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-certs\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866353    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-credential-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866444    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866560    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-data\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.866695 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.866676    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91745d21-1539-4c6a-a38d-250b69eb48ef-istiod-ca-cert\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.868358 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.868331    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-envoy\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.868475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.868361    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-podinfo\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.927992 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.927964    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-token\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.928332 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.928314    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8d4h\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-kube-api-access-f8d4h\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:43.961896 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:43.961837    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:44.043607 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.043572    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:44.253399 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.253280    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:44.270071 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.270044    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"]
Nov 03 22:32:44.273785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.273769    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.327920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.327898    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-5e72ad-openshift-default-dockercfg-5qnwn\""
Nov 03 22:32:44.371080 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371055    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-podinfo\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371196 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371107    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-credential-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371187    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-data\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371214    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-envoy\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371322 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371245    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-token\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371365 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371347    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371373    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-certs\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371496    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77lsk\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-kube-api-access-77lsk\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.371548 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.371534    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istiod-ca-cert\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.465357 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.465321    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:44.472410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472363    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472408    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-certs\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472459    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77lsk\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-kube-api-access-77lsk\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472496    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istiod-ca-cert\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472541    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-podinfo\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472712 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472584    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-credential-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472712 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472614    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-data\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472712 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472640    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-envoy\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472740    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-token\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472816    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.472997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.472964    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-data\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.473059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.473012    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-certs\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.473107 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.473052    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-credential-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.473372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.473345    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istiod-ca-cert\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.474950 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.474929    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-envoy\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.475079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.475063    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-podinfo\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.538104 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.538048    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-token\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.538104 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.538079    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77lsk\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-kube-api-access-77lsk\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.586670 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.586630    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:44.672120 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.672075    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:44.887236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.887011    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"]
Nov 03 22:32:44.890424 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.890367    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:44.892249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.892231    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.954019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.953986    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-68fbd1-openshift-default-dockercfg-sd2d7\""
Nov 03 22:32:44.979076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979049    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2f2k\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-kube-api-access-f2f2k\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979178    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979315    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a00c0-b9af-420d-9b3f-cf928586bff2-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979373    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979424    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979459    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979483    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979545    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:44.979716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:44.979637    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.080580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080546    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.080580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080582    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.080781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080599    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.080781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080655    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.080781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080731    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2f2k\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-kube-api-access-f2f2k\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.080781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080766    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081021 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080999    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081075 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.081017    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081075 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.080999    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a00c0-b9af-420d-9b3f-cf928586bff2-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081075 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.081073    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.081082    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.081091    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081452 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.081432    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.081682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.081658    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a00c0-b9af-420d-9b3f-cf928586bff2-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.084321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.084290    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.084553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.084537    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.084882 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.084863    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"]
Nov 03 22:32:45.093924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.093903    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"]
Nov 03 22:32:45.141908 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.141835    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"]
Nov 03 22:32:45.146841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.146813    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2f2k\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-kube-api-access-f2f2k\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.147340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.147320    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.187883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.187850    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:45.205577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.205554    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:45.395557 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.395464    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:45.576666 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.576631    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"]
Nov 03 22:32:45.590563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.590527    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:45.608833 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:32:45.608811    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69aa4032_5ee5_4b3e_95c2_1e39feb71bcc.slice/crio-b1db083719adf6758ccbbe1cbdcf29e5242714092b7f3d11bda89c1a96d10de8 WatchSource:0}: Error finding container b1db083719adf6758ccbbe1cbdcf29e5242714092b7f3d11bda89c1a96d10de8: Status 404 returned error can't find the container with id b1db083719adf6758ccbbe1cbdcf29e5242714092b7f3d11bda89c1a96d10de8
Nov 03 22:32:45.610868 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.610841    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.610928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.610901    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.611006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.610932    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.624998 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:32:45.624973    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395a00c0_b9af_420d_9b3f_cf928586bff2.slice/crio-3f8f4d0666c706e7284a36e43797688c674a097bb7ef979eb191d0db47cfe399 WatchSource:0}: Error finding container 3f8f4d0666c706e7284a36e43797688c674a097bb7ef979eb191d0db47cfe399: Status 404 returned error can't find the container with id 3f8f4d0666c706e7284a36e43797688c674a097bb7ef979eb191d0db47cfe399
Nov 03 22:32:45.627251 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.627215    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.627327 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.627308    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.627435 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.627384    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.627518 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.627424    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"]
Nov 03 22:32:45.670552 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:32:45.670520    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91745d21_1539_4c6a_a38d_250b69eb48ef.slice/crio-2fd05cc603dc1d29b0b99472b05b33d895062d7813bb921f43b7d2d85f82e469 WatchSource:0}: Error finding container 2fd05cc603dc1d29b0b99472b05b33d895062d7813bb921f43b7d2d85f82e469: Status 404 returned error can't find the container with id 2fd05cc603dc1d29b0b99472b05b33d895062d7813bb921f43b7d2d85f82e469
Nov 03 22:32:45.672891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.672857    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.673006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.672919    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.673006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.672947    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:45.674856 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.674831    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"]
Nov 03 22:32:45.795497 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.795442    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:45.844081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.844046    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" event={"ID":"395a00c0-b9af-420d-9b3f-cf928586bff2","Type":"ContainerStarted","Data":"952c60156a37c0def551a62b0025fad0cfe7899ebfb362e5ed9bfd96dc32d876"}
Nov 03 22:32:45.844332 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.844304    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" event={"ID":"395a00c0-b9af-420d-9b3f-cf928586bff2","Type":"ContainerStarted","Data":"3f8f4d0666c706e7284a36e43797688c674a097bb7ef979eb191d0db47cfe399"}
Nov 03 22:32:45.846504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.846479    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" event={"ID":"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc","Type":"ContainerStarted","Data":"5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09"}
Nov 03 22:32:45.846973 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.846952    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" event={"ID":"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc","Type":"ContainerStarted","Data":"b1db083719adf6758ccbbe1cbdcf29e5242714092b7f3d11bda89c1a96d10de8"}
Nov 03 22:32:45.848183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.848161    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" event={"ID":"91745d21-1539-4c6a-a38d-250b69eb48ef","Type":"ContainerStarted","Data":"a905ec820d03d51e7b246db1f45437c187409a94e2c957a4b42193040da760c6"}
Nov 03 22:32:45.848297 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.848202    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" event={"ID":"91745d21-1539-4c6a-a38d-250b69eb48ef","Type":"ContainerStarted","Data":"2fd05cc603dc1d29b0b99472b05b33d895062d7813bb921f43b7d2d85f82e469"}
Nov 03 22:32:45.962638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.962609    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:45.972563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:45.972355    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podStartSLOduration=1.9723357259999998 podStartE2EDuration="1.972335726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:45.966727136 +0000 UTC m=+569.231427147" watchObservedRunningTime="2026-11-03 22:32:45.972335726 +0000 UTC m=+569.237035739"
Nov 03 22:32:46.091879 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.091824    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podStartSLOduration=2.091811588 podStartE2EDuration="2.091811588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:46.089787911 +0000 UTC m=+569.354487922" watchObservedRunningTime="2026-11-03 22:32:46.091811588 +0000 UTC m=+569.356511649"
Nov 03 22:32:46.208309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.208102    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:46.221499 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.221301    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podStartSLOduration=3.221217033 podStartE2EDuration="3.221217033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:46.219810305 +0000 UTC m=+569.484510317" watchObservedRunningTime="2026-11-03 22:32:46.221217033 +0000 UTC m=+569.485917042"
Nov 03 22:32:46.222339 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.222245    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:46.587047 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.586962    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:46.592301 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.592278    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:46.854300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.854220    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:46.854300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.854266    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:46.854943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.854920    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:32:46.855052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.854971    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:32:46.967265 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:46.967241    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:47.858911 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:47.858879    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:47.859747 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:47.859729    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:32:49.076519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:49.076486    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:49.278188 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:49.278155    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:49.684329 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:49.684292    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:49.949772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:49.949669    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:50.288977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:50.288869    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:50.587249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:50.587164    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:52.521198 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:52.521162    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:52.773486 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:52.773363    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:53.626068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.626033    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"]
Nov 03 22:32:53.629845 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.629829    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667088    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667272 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667142    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9233afc-7975-4d25-b869-612a49d4038a-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667272 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667251    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667368    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667478 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667459    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667536 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667481    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9233afc-7975-4d25-b869-612a49d4038a-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667595 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667562    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667665 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667646    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clnn7\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-kube-api-access-clnn7\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.667727 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.667698    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.684239 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.684217    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-18c03a-openshift-default-dockercfg-p87mq\""
Nov 03 22:32:53.737961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.737935    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"]
Nov 03 22:32:53.769131 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769101    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769149    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769170    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9233afc-7975-4d25-b869-612a49d4038a-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769320    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769508 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769450    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clnn7\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-kube-api-access-clnn7\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769574    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769612    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769696    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769764    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9233afc-7975-4d25-b869-612a49d4038a-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769790    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.769964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.769817    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.770344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.770305    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.770477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.770375    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9233afc-7975-4d25-b869-612a49d4038a-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.770602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.770583    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.774953 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.774916    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:53.780010 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.779990    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9233afc-7975-4d25-b869-612a49d4038a-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.780952 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.780936    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.831975 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.831905    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clnn7\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-kube-api-access-clnn7\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.843014 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.842966    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:53.943199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:53.943164    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:54.036449 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.036412    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:54.364128 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:32:54.364096    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9233afc_7975_4d25_b869_612a49d4038a.slice/crio-73d5dd67356775109a403fe52b4064d2af03e41fbd1e1bb4a6fc6697b7746998 WatchSource:0}: Error finding container 73d5dd67356775109a403fe52b4064d2af03e41fbd1e1bb4a6fc6697b7746998: Status 404 returned error can't find the container with id 73d5dd67356775109a403fe52b4064d2af03e41fbd1e1bb4a6fc6697b7746998
Nov 03 22:32:54.366651 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.366614    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:54.366733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.366716    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:54.366776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.366757    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:32:54.367116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.367095    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"]
Nov 03 22:32:54.895669 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.895634    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" event={"ID":"b9233afc-7975-4d25-b869-612a49d4038a","Type":"ContainerStarted","Data":"2d68b272f480080d36b2359bf1cb3dc9afda3f76fc689b54431ab4d205f1ff65"}
Nov 03 22:32:54.896026 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.895691    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" event={"ID":"b9233afc-7975-4d25-b869-612a49d4038a","Type":"ContainerStarted","Data":"73d5dd67356775109a403fe52b4064d2af03e41fbd1e1bb4a6fc6697b7746998"}
Nov 03 22:32:54.941207 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.941180    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"]
Nov 03 22:32:54.944783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.944768    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:54.944860 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.944850    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:54.981174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.981151    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-metrics-certs\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:54.981283 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.981224    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fe8f39-c007-4989-bdba-7ef33abe9647-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:54.981283 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.981258    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:54.981355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.981316    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9h95\" (UniqueName: \"kubernetes.io/projected/11fe8f39-c007-4989-bdba-7ef33abe9647-kube-api-access-j9h95\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:54.998796 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:54.998777    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-6f1190-serving-cert\""
Nov 03 22:32:55.000306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.000281    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"]
Nov 03 22:32:55.061023 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.060975    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" podStartSLOduration=2.060963788 podStartE2EDuration="2.060963788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:55.059161712 +0000 UTC m=+578.323861723" watchObservedRunningTime="2026-11-03 22:32:55.060963788 +0000 UTC m=+578.325663798"
Nov 03 22:32:55.082199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.082174    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.082349 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.082255    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9h95\" (UniqueName: \"kubernetes.io/projected/11fe8f39-c007-4989-bdba-7ef33abe9647-kube-api-access-j9h95\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.082349 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:32:55.082299    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-serving-cert: secret "maas-api-e2e-worker-w5-6f1190-serving-cert" not found
Nov 03 22:32:55.082504 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:32:55.082371    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls podName:11fe8f39-c007-4989-bdba-7ef33abe9647 nodeName:}" failed. No retries permitted until 2026-11-03 22:32:55.582351429 +0000 UTC m=+578.847051433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls") pod "maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd" (UID: "11fe8f39-c007-4989-bdba-7ef33abe9647") : secret "maas-api-e2e-worker-w5-6f1190-serving-cert" not found
Nov 03 22:32:55.082504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.082418    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-metrics-certs\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.082504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.082487    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fe8f39-c007-4989-bdba-7ef33abe9647-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.083151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.083129    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fe8f39-c007-4989-bdba-7ef33abe9647-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.084869 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.084847    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-metrics-certs\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.145961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.145905    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9h95\" (UniqueName: \"kubernetes.io/projected/11fe8f39-c007-4989-bdba-7ef33abe9647-kube-api-access-j9h95\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.186623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.184025    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:55.386441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.386374    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:55.587286 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.587258    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.589617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.589594    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls\") pod \"maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.659321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.659296    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:55.857614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.857547    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:55.878092 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.878056    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:55.949166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:55.949142    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:56.231592 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:32:56.231562    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11fe8f39_c007_4989_bdba_7ef33abe9647.slice/crio-df808b4ef8b1fcf61aab5310e97f1a5f36038416ce82cbdce0ce2ab34c6cc7de WatchSource:0}: Error finding container df808b4ef8b1fcf61aab5310e97f1a5f36038416ce82cbdce0ce2ab34c6cc7de: Status 404 returned error can't find the container with id df808b4ef8b1fcf61aab5310e97f1a5f36038416ce82cbdce0ce2ab34c6cc7de
Nov 03 22:32:56.233906 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:56.233755    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"]
Nov 03 22:32:56.241752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:56.241726    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:56.546030 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:56.545933    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:56.914250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:56.914174    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd" event={"ID":"11fe8f39-c007-4989-bdba-7ef33abe9647","Type":"ContainerStarted","Data":"df808b4ef8b1fcf61aab5310e97f1a5f36038416ce82cbdce0ce2ab34c6cc7de"}
Nov 03 22:32:56.914403 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:56.914360    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:56.915316 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:56.915295    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:32:57.425597 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:57.425560    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:57.674657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:57.674624    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:32:57.922174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:57.922136    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd" event={"ID":"11fe8f39-c007-4989-bdba-7ef33abe9647","Type":"ContainerStarted","Data":"c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9"}
Nov 03 22:32:57.922679 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:57.922558    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:32:58.047159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:58.047069    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd" podStartSLOduration=3.137774525 podStartE2EDuration="4.047054302s" totalImagesPullingTime="909.279777ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:58.046630105 +0000 UTC m=+581.311330129" watchObservedRunningTime="2026-11-03 22:32:58.047054302 +0000 UTC m=+581.311754313"
Nov 03 22:32:59.932158 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:32:59.932117    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:00.132753 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:00.132714    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:00.337310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:00.337218    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:00.544577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:00.544543    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:02.106922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:02.106877    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:02.362906 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:02.362817    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:02.564535 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:02.564503    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:02.770210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:02.770166    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:03.932507 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:03.932483    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 22:33:04.158989 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:04.158957    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:04.408610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:04.408526    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:05.245992 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:05.245958    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:05.498184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:05.498105    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:07.525009 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.524970    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:07.722960 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.722927    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:07.859962 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.859882    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"]
Nov 03 22:33:07.863646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.863621    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:07.916417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.916356    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07b6e3a4-895b-4344-8897-22455346de35-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:07.916598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.916452    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9wkt\" (UniqueName: \"kubernetes.io/projected/07b6e3a4-895b-4344-8897-22455346de35-kube-api-access-k9wkt\") pod \"payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:07.917584 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.917558    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"]
Nov 03 22:33:07.966488 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.966455    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-961054\""
Nov 03 22:33:07.966658 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:07.966497    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-961054-dockercfg-6kmx7\""
Nov 03 22:33:08.017257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:08.017225    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9wkt\" (UniqueName: \"kubernetes.io/projected/07b6e3a4-895b-4344-8897-22455346de35-kube-api-access-k9wkt\") pod \"payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:08.017451 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:08.017357    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07b6e3a4-895b-4344-8897-22455346de35-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:08.017981 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:08.017963    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07b6e3a4-895b-4344-8897-22455346de35-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:08.089897 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:08.089860    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9wkt\" (UniqueName: \"kubernetes.io/projected/07b6e3a4-895b-4344-8897-22455346de35-kube-api-access-k9wkt\") pod \"payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:08.176061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:08.176038    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:08.544112 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:33:08.544081    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b6e3a4_895b_4344_8897_22455346de35.slice/crio-27626ae9a58a256588d18d93d01d854ffc450080f0c7f92cc382f9052c84a205 WatchSource:0}: Error finding container 27626ae9a58a256588d18d93d01d854ffc450080f0c7f92cc382f9052c84a205: Status 404 returned error can't find the container with id 27626ae9a58a256588d18d93d01d854ffc450080f0c7f92cc382f9052c84a205
Nov 03 22:33:08.547214 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:08.547193    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"]
Nov 03 22:33:09.001826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:09.000951    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" event={"ID":"07b6e3a4-895b-4344-8897-22455346de35","Type":"ContainerStarted","Data":"27626ae9a58a256588d18d93d01d854ffc450080f0c7f92cc382f9052c84a205"}
Nov 03 22:33:09.927517 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:09.927483    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"]
Nov 03 22:33:09.931078 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:09.931056    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:09.986999 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:09.986964    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"]
Nov 03 22:33:10.035362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.035330    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-961054-serving-cert\""
Nov 03 22:33:10.037815 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.037794    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.037924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.037834    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq7vh\" (UniqueName: \"kubernetes.io/projected/d895f971-4533-4101-b67b-1fec8a211ece-kube-api-access-hq7vh\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.037924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.037855    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d895f971-4533-4101-b67b-1fec8a211ece-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.038201 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.038176    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-metrics-certs\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.090430 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:33:10.090374    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:33:10.138782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.138759    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-metrics-certs\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.138888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.138854    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.138936 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.138899    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq7vh\" (UniqueName: \"kubernetes.io/projected/d895f971-4533-4101-b67b-1fec8a211ece-kube-api-access-hq7vh\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.139018 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:33:10.138988    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-serving-cert: secret "maas-api-e2e-worker-w3-961054-serving-cert" not found
Nov 03 22:33:10.139140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.139024    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d895f971-4533-4101-b67b-1fec8a211ece-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.139140 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:33:10.139063    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls podName:d895f971-4533-4101-b67b-1fec8a211ece nodeName:}" failed. No retries permitted until 2026-11-03 22:33:10.639043285 +0000 UTC m=+593.903743280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls") pod "maas-api-e2e-worker-w3-961054-86d699db45-47wpk" (UID: "d895f971-4533-4101-b67b-1fec8a211ece") : secret "maas-api-e2e-worker-w3-961054-serving-cert" not found
Nov 03 22:33:10.139711 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.139695    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d895f971-4533-4101-b67b-1fec8a211ece-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.141348 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.141330    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-metrics-certs\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.174899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.174857    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:10.204154 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.204092    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq7vh\" (UniqueName: \"kubernetes.io/projected/d895f971-4533-4101-b67b-1fec8a211ece-kube-api-access-hq7vh\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.476800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.476547    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:10.645985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.645954    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.648657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.648632    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls\") pod \"maas-api-e2e-worker-w3-961054-86d699db45-47wpk\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:10.847594 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:10.847520    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:11.228319 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:33:11.228282    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd895f971_4533_4101_b67b_1fec8a211ece.slice/crio-3c79a3d166017e3b6c9ec7a91cab55da42bc4b612445cd45c27b72bca63f1a19 WatchSource:0}: Error finding container 3c79a3d166017e3b6c9ec7a91cab55da42bc4b612445cd45c27b72bca63f1a19: Status 404 returned error can't find the container with id 3c79a3d166017e3b6c9ec7a91cab55da42bc4b612445cd45c27b72bca63f1a19
Nov 03 22:33:11.232178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:11.232147    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"]
Nov 03 22:33:11.237141 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:11.237113    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:12.019578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:12.019543    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" event={"ID":"07b6e3a4-895b-4344-8897-22455346de35","Type":"ContainerStarted","Data":"38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84"}
Nov 03 22:33:12.019732 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:12.019614    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:12.020653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:12.020634    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk" event={"ID":"d895f971-4533-4101-b67b-1fec8a211ece","Type":"ContainerStarted","Data":"3c79a3d166017e3b6c9ec7a91cab55da42bc4b612445cd45c27b72bca63f1a19"}
Nov 03 22:33:12.142192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:12.142143    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" podStartSLOduration=2.128448843 podStartE2EDuration="5.142130146s" totalImagesPullingTime="3.013681303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:12.141690293 +0000 UTC m=+595.406390304" watchObservedRunningTime="2026-11-03 22:33:12.142130146 +0000 UTC m=+595.406830158"
Nov 03 22:33:13.028242 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:13.028198    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk" event={"ID":"d895f971-4533-4101-b67b-1fec8a211ece","Type":"ContainerStarted","Data":"bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58"}
Nov 03 22:33:13.028737 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:13.028270    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:13.155589 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:13.155536    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk" podStartSLOduration=3.17681944 podStartE2EDuration="4.155520589s" totalImagesPullingTime="978.701149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:13.1540437 +0000 UTC m=+596.418743710" watchObservedRunningTime="2026-11-03 22:33:13.155520589 +0000 UTC m=+596.420220600"
Nov 03 22:33:14.784272 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.784239    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"]
Nov 03 22:33:14.788036 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.788020    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:14.867416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.859407    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-5e72ad-serving-cert\""
Nov 03 22:33:14.871863 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.871825    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:14.914422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.913700    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59b8b\" (UniqueName: \"kubernetes.io/projected/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-kube-api-access-59b8b\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:14.914422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.913808    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:14.914422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.913846    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-metrics-certs\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:14.914422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.913918    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:14.920873 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:14.920842    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"]
Nov 03 22:33:15.015488 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.015460    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.015642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.015584    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59b8b\" (UniqueName: \"kubernetes.io/projected/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-kube-api-access-59b8b\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.015684 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.015661    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.015720 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.015693    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-metrics-certs\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.015821 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:33:15.015800    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-serving-cert: secret "maas-api-e2e-worker-w6-5e72ad-serving-cert" not found
Nov 03 22:33:15.015882 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:33:15.015871    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls podName:46dcc5d8-b2d9-4c60-9e76-8cf25739094b nodeName:}" failed. No retries permitted until 2026-11-03 22:33:15.515852782 +0000 UTC m=+598.780552790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls") pod "maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh" (UID: "46dcc5d8-b2d9-4c60-9e76-8cf25739094b") : secret "maas-api-e2e-worker-w6-5e72ad-serving-cert" not found
Nov 03 22:33:15.016264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.016248    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.018011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.017995    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-metrics-certs\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.083987 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.083913    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59b8b\" (UniqueName: \"kubernetes.io/projected/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-kube-api-access-59b8b\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.105794 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.105767    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:15.375100 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.375000    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:15.527283 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.527251    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.529751 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.529727    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:15.701336 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:15.701302    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:16.094459 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:33:16.094425    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46dcc5d8_b2d9_4c60_9e76_8cf25739094b.slice/crio-c5f1ef844710c4fc7dad768dd772a2c6c4ec945d93430f1c1199378c5ea367fc WatchSource:0}: Error finding container c5f1ef844710c4fc7dad768dd772a2c6c4ec945d93430f1c1199378c5ea367fc: Status 404 returned error can't find the container with id c5f1ef844710c4fc7dad768dd772a2c6c4ec945d93430f1c1199378c5ea367fc
Nov 03 22:33:16.097205 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:16.097184    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"]
Nov 03 22:33:17.073659 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:17.073628    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh" event={"ID":"46dcc5d8-b2d9-4c60-9e76-8cf25739094b","Type":"ContainerStarted","Data":"c5f1ef844710c4fc7dad768dd772a2c6c4ec945d93430f1c1199378c5ea367fc"}
Nov 03 22:33:17.419510 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:17.419473    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:17.688196 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:17.683189    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:18.080378 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:18.080295    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh" event={"ID":"46dcc5d8-b2d9-4c60-9e76-8cf25739094b","Type":"ContainerStarted","Data":"9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897"}
Nov 03 22:33:18.080550 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:18.080432    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:18.207298 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:18.207252    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh" podStartSLOduration=3.272910466 podStartE2EDuration="4.207238438s" totalImagesPullingTime="934.327972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:18.205301551 +0000 UTC m=+601.470001561" watchObservedRunningTime="2026-11-03 22:33:18.207238438 +0000 UTC m=+601.471938448"
Nov 03 22:33:19.038770 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:19.038744    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 22:33:19.834298 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:19.834256    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:23.032297 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:23.032211    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 22:33:23.116029 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:23.115952    5162 generic.go:356] "Generic (PLEG): container finished" podID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerID="952c60156a37c0def551a62b0025fad0cfe7899ebfb362e5ed9bfd96dc32d876" exitCode=0
Nov 03 22:33:23.116172 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:23.116021    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" event={"ID":"395a00c0-b9af-420d-9b3f-cf928586bff2","Type":"ContainerDied","Data":"952c60156a37c0def551a62b0025fad0cfe7899ebfb362e5ed9bfd96dc32d876"}
Nov 03 22:33:23.116423 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:23.116384    5162 scope.go:122] "RemoveContainer" containerID="952c60156a37c0def551a62b0025fad0cfe7899ebfb362e5ed9bfd96dc32d876"
Nov 03 22:33:23.117735 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:23.117707    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:23.117844 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:23.117788    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:23.117844 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:23.117837    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:24.091139 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:24.091114    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 22:33:24.124748 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:24.124709    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" event={"ID":"395a00c0-b9af-420d-9b3f-cf928586bff2","Type":"ContainerStarted","Data":"23702f64da24864b7cccff92b1ac4010383e9a55e92a13b530dbedbc8a01e907"}
Nov 03 22:33:24.206651 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:24.206618    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:33:24.206836 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:24.206815    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:33:24.206906 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:24.206888    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:33:24.435798 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:24.435771    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:25.208126 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:25.207842    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:33:25.208126 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:25.207905    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:33:26.214440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:26.207772    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:33:26.214440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:26.207832    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:33:27.207859 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:27.207818    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:33:27.208169 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:27.208131    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:33:28.209415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:28.208769    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:33:28.209415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:28.208831    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:33:28.307416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:28.307364    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:28.908864 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:28.908771    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:29.118624 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:29.118580    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:29.208414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:29.208005    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:33:29.208414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:29.208101    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:33:29.431882 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:29.431845    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:30.210504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:30.210472    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:33:30.210962 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:30.210918    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:33:30.212108 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:30.212088    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 22:33:32.895538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:32.895497    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:33:32.896000 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:32.895557    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:33:33.152154 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.152054    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:58250->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Nov 03 22:33:33.152154 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.152141    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:58250->10.134.0.45:15021: read: connection reset by peer"
Nov 03 22:33:33.236762 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.236735    5162 generic.go:356] "Generic (PLEG): container finished" podID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerID="a905ec820d03d51e7b246db1f45437c187409a94e2c957a4b42193040da760c6" exitCode=0
Nov 03 22:33:33.236874 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.236783    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" event={"ID":"91745d21-1539-4c6a-a38d-250b69eb48ef","Type":"ContainerDied","Data":"a905ec820d03d51e7b246db1f45437c187409a94e2c957a4b42193040da760c6"}
Nov 03 22:33:33.237166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.237148    5162 scope.go:122] "RemoveContainer" containerID="a905ec820d03d51e7b246db1f45437c187409a94e2c957a4b42193040da760c6"
Nov 03 22:33:33.238726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.238691    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:33.238810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.238782    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:33.238850 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.238823    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:33.790810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:33.790776    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:34.040777 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.040694    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:34.267307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.267269    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:34.267530 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.267509    5162 generic.go:356] "Generic (PLEG): container finished" podID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerID="5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09" exitCode=0
Nov 03 22:33:34.267613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.267588    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" event={"ID":"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc","Type":"ContainerDied","Data":"5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09"}
Nov 03 22:33:34.268070 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.268048    5162 scope.go:122] "RemoveContainer" containerID="5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09"
Nov 03 22:33:34.269614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.269583    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:34.269729 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.269668    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:34.269729 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.269720    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:34.269921 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.269900    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" event={"ID":"91745d21-1539-4c6a-a38d-250b69eb48ef","Type":"ContainerStarted","Data":"ad9fa409ab7df5b20e323c0794855bb326e0cd66dc9ecf131bbab06b2c2ef45e"}
Nov 03 22:33:34.965416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.964830    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:33:34.965416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.964967    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 03 22:33:34.965416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:34.965013    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 03 22:33:35.303886 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:35.303800    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" event={"ID":"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc","Type":"ContainerStarted","Data":"48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d"}
Nov 03 22:33:35.591464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:35.590790    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:33:35.591464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:35.590915    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 03 22:33:35.591464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:35.590961    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 03 22:33:35.963415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:35.962796    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 03 22:33:35.963415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:35.962859    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 03 22:33:36.589910 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:36.589861    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 03 22:33:36.590499 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:36.589931    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 03 22:33:36.962787 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:36.962746    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 03 22:33:36.963073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:36.963043    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 03 22:33:37.589427 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:37.588847    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 03 22:33:37.589427 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:37.588910    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 03 22:33:37.969419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:37.963846    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 03 22:33:37.969419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:37.963912    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 03 22:33:38.394482 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:38.391232    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:38.587912 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:38.587876    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 03 22:33:38.588230 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:38.588187    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 03 22:33:38.965810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:38.965773    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 03 22:33:38.966118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:38.966087    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 03 22:33:39.587143 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:39.587113    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Nov 03 22:33:39.587576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:39.587160    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Nov 03 22:33:39.969252 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:39.969220    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:33:39.969804 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:39.969784    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:33:39.970697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:39.970677    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 22:33:40.077189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:40.077154    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:40.092956 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:33:40.092928    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:33:40.330480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:40.330403    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:40.590718 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:40.590645    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:33:40.591076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:40.590970    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:33:40.591652 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:40.591636    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:33:42.883331 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:42.883295    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:43.803163 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.803132    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"]
Nov 03 22:33:43.805968 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.805952    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.816916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.816893    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tls-certs\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.817015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.816931    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-home\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.817063 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.817050    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tmp-dir\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.817102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.817081    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-model-cache\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.817102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.817097    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qxc\" (UniqueName: \"kubernetes.io/projected/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-kube-api-access-f4qxc\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.817206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.817191    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-dshm\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.915480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.915451    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-6f1190\"/\"openshift-service-ca.crt\""
Nov 03 22:33:43.915846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.915521    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-6f1190\"/\"simulator-w5-6f1190-kserve-self-signed-certs\""
Nov 03 22:33:43.915846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.915541    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-6f1190\"/\"default-dockercfg-k9b9h\""
Nov 03 22:33:43.915846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.915588    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-6f1190\"/\"kube-root-ca.crt\""
Nov 03 22:33:43.918009 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.917990    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"]
Nov 03 22:33:43.918098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918082    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tmp-dir\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918146 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918116    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-model-cache\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918146 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918135    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qxc\" (UniqueName: \"kubernetes.io/projected/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-kube-api-access-f4qxc\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918156    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-dshm\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918192    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tls-certs\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918224    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-home\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918532    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tmp-dir\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918612 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918555    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-model-cache\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.918612 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.918602    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-home\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.920369 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.920345    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-dshm\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.920627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.920611    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tls-certs\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.981205 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.981175    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qxc\" (UniqueName: \"kubernetes.io/projected/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-kube-api-access-f4qxc\") pod \"simulator-w5-6f1190-kserve-6885f5f869-lm2dr\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") " pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:43.981974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:43.981947    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:44.117480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:44.117413    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:44.493577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:44.493543    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"]
Nov 03 22:33:44.493902 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:33:44.493875    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf923fc18_ec7e_4028_865c_c6e0f1b40f3f.slice/crio-43e407a5fdd21fbcaf803e24a348b3c8dbdaf8ddf628694f3b6512978f3cae62 WatchSource:0}: Error finding container 43e407a5fdd21fbcaf803e24a348b3c8dbdaf8ddf628694f3b6512978f3cae62: Status 404 returned error can't find the container with id 43e407a5fdd21fbcaf803e24a348b3c8dbdaf8ddf628694f3b6512978f3cae62
Nov 03 22:33:45.386465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:45.386429    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" event={"ID":"f923fc18-ec7e-4028-865c-c6e0f1b40f3f","Type":"ContainerStarted","Data":"d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c"}
Nov 03 22:33:45.386883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:45.386475    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" event={"ID":"f923fc18-ec7e-4028-865c-c6e0f1b40f3f","Type":"ContainerStarted","Data":"43e407a5fdd21fbcaf803e24a348b3c8dbdaf8ddf628694f3b6512978f3cae62"}
Nov 03 22:33:45.536760 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:45.536702    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podStartSLOduration=2.536685662 podStartE2EDuration="2.536685662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:45.534767239 +0000 UTC m=+628.799467249" watchObservedRunningTime="2026-11-03 22:33:45.536685662 +0000 UTC m=+628.801385705"
Nov 03 22:33:45.693119 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:45.693080    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:45.895952 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:45.895908    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:46.096906 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:46.096816    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:46.117667 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:46.117642    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:46.307020 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:46.306979    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:46.514535 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:46.514503    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:47.799589 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:47.799558    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:52.572077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:52.572039    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:53.122860 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.122826    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:53.433639 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.433604    5162 generic.go:356] "Generic (PLEG): container finished" podID="b9233afc-7975-4d25-b869-612a49d4038a" containerID="2d68b272f480080d36b2359bf1cb3dc9afda3f76fc689b54431ab4d205f1ff65" exitCode=0
Nov 03 22:33:53.433842 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.433678    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" event={"ID":"b9233afc-7975-4d25-b869-612a49d4038a","Type":"ContainerDied","Data":"2d68b272f480080d36b2359bf1cb3dc9afda3f76fc689b54431ab4d205f1ff65"}
Nov 03 22:33:53.434121 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.434101    5162 scope.go:122] "RemoveContainer" containerID="2d68b272f480080d36b2359bf1cb3dc9afda3f76fc689b54431ab4d205f1ff65"
Nov 03 22:33:53.435578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.435547    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:53.435643 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.435616    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:53.435679 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.435664    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:33:53.477627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.477595    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:53.747123 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.747041    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:53.968977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:53.968918    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:54.117897 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.117817    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:54.130876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.130854    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:54.174520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.174488    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:54.441023 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.440991    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" event={"ID":"b9233afc-7975-4d25-b869-612a49d4038a","Type":"ContainerStarted","Data":"d817f356aa9448197e72934311f2662c932f23f656d8eb9c9b7d323d82e6db9d"}
Nov 03 22:33:54.476656 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.476622    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:33:54.965643 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.965611    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:33:54.966938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.966914    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Nov 03 22:33:54.967035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:54.966962    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Nov 03 22:33:55.954461 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:55.953730    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Nov 03 22:33:55.954461 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:55.953790    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Nov 03 22:33:56.915137 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:56.915103    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:33:56.946721 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:56.946688    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Nov 03 22:33:56.946992 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:56.946963    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Nov 03 22:33:57.691042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:57.691008    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:33:57.965612 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:57.965523    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Nov 03 22:33:57.965612 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:57.965595    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Nov 03 22:33:58.956131 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:58.955711    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Nov 03 22:33:58.956131 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:58.955770    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Nov 03 22:33:59.948233 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:59.948206    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:33:59.949245 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:33:59.949225    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 22:34:00.317754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:00.317679    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:00.937887 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:00.937859    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:01.143613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:01.143585    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:02.549000 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:02.548958    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:07.280979 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.280900    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:07.474634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.474597    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"]
Nov 03 22:34:07.477532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.477517    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.533279 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.533190    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"]
Nov 03 22:34:07.581631 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.581599    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ee74cb\"/\"kube-root-ca.crt\""
Nov 03 22:34:07.581795 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.581640    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ee74cb\"/\"openshift-service-ca.crt\""
Nov 03 22:34:07.581795 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.581640    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ee74cb\"/\"simulator-w2-ee74cb-kserve-self-signed-certs\""
Nov 03 22:34:07.583335 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.583320    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ee74cb\"/\"default-dockercfg-ptcqx\""
Nov 03 22:34:07.627096 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.627061    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-dshm\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.627270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.627115    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-home\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.627270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.627183    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-tmp-dir\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.627270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.627206    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96987f24-3f13-4d94-8754-208bbeb8844e-tls-certs\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.627270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.627261    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh6gp\" (UniqueName: \"kubernetes.io/projected/96987f24-3f13-4d94-8754-208bbeb8844e-kube-api-access-zh6gp\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.627433 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.627410    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-model-cache\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.630184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.630155    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:07.728520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728495    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-model-cache\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.728675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728582    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-dshm\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.728675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728612    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-home\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.728675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728630    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-tmp-dir\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.728675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728649    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96987f24-3f13-4d94-8754-208bbeb8844e-tls-certs\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.728832 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728715    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh6gp\" (UniqueName: \"kubernetes.io/projected/96987f24-3f13-4d94-8754-208bbeb8844e-kube-api-access-zh6gp\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.728965 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728945    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-model-cache\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.729002 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.728990    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-tmp-dir\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.729070 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.729047    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-home\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.730997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.730978    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-dshm\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.731171 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.731153    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96987f24-3f13-4d94-8754-208bbeb8844e-tls-certs\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:07.791259 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:07.791196    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh6gp\" (UniqueName: \"kubernetes.io/projected/96987f24-3f13-4d94-8754-208bbeb8844e-kube-api-access-zh6gp\") pod \"simulator-w2-ee74cb-kserve-5554b74f97-zktr5\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") " pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:08.089112 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:08.089025    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:08.458408 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:34:08.458360    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96987f24_3f13_4d94_8754_208bbeb8844e.slice/crio-0f2302d9204cfe41cb970f91aa66361eecc546f645dd90da8ddb30255658fd13 WatchSource:0}: Error finding container 0f2302d9204cfe41cb970f91aa66361eecc546f645dd90da8ddb30255658fd13: Status 404 returned error can't find the container with id 0f2302d9204cfe41cb970f91aa66361eecc546f645dd90da8ddb30255658fd13
Nov 03 22:34:08.461535 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:08.461510    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"]
Nov 03 22:34:08.547643 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:08.547607    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" event={"ID":"96987f24-3f13-4d94-8754-208bbeb8844e","Type":"ContainerStarted","Data":"fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5"}
Nov 03 22:34:08.547789 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:08.547657    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" event={"ID":"96987f24-3f13-4d94-8754-208bbeb8844e","Type":"ContainerStarted","Data":"0f2302d9204cfe41cb970f91aa66361eecc546f645dd90da8ddb30255658fd13"}
Nov 03 22:34:08.681001 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:08.680951    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podStartSLOduration=1.6809379660000001 podStartE2EDuration="1.680937966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:08.680119167 +0000 UTC m=+651.944819177" watchObservedRunningTime="2026-11-03 22:34:08.680937966 +0000 UTC m=+651.945637976"
Nov 03 22:34:09.089122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:09.089092    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:10.082992 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:34:10.082953    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:34:11.889050 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:11.888999    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:13.557416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:13.556382    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:14.017692 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:14.017661    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:14.654112 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:14.654079    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:14.919934 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:14.919908    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:16.768162 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:16.768124    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:18.089501 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:18.089462    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:18.101975 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:18.101951    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:18.619362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:18.619333    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:34:20.503824 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:20.503784    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:21.384192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:21.384157    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:22.238914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:22.238854    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:25.879232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:25.879196    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:30.401826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:30.401784    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:31.121943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:31.121906    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:31.335638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:31.335608    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:32.460442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.460410    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"]
Nov 03 22:34:32.464117 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.464100    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.517545 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.517513    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-tmp-dir\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.517545 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.517546    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9ac93-5f42-48c8-85a0-848d53fa5056-tls-certs\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.517774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.517574    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-home\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.517774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.517693    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-dshm\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.517883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.517786    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-model-cache\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.517883 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.517817    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp7jd\" (UniqueName: \"kubernetes.io/projected/ede9ac93-5f42-48c8-85a0-848d53fa5056-kube-api-access-kp7jd\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.519025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.519001    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"]
Nov 03 22:34:32.567920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.567883    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-68fbd1\"/\"openshift-service-ca.crt\""
Nov 03 22:34:32.567920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.567914    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-68fbd1\"/\"kube-root-ca.crt\""
Nov 03 22:34:32.568096 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.567917    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-68fbd1\"/\"default-dockercfg-5blrx\""
Nov 03 22:34:32.568096 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.567979    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-68fbd1\"/\"simulator-w0-68fbd1-kserve-self-signed-certs\""
Nov 03 22:34:32.599600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.599567    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:32.619149 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619107    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-tmp-dir\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.619149 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619151    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9ac93-5f42-48c8-85a0-848d53fa5056-tls-certs\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.619386 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619188    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-home\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.619386 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619214    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-dshm\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.619386 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619295    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-model-cache\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.619386 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619333    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp7jd\" (UniqueName: \"kubernetes.io/projected/ede9ac93-5f42-48c8-85a0-848d53fa5056-kube-api-access-kp7jd\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.619627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619581    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-tmp-dir\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.619889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.619852    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-home\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.620299 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.620276    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-model-cache\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.622275 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.622252    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-dshm\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.623743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.623723    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9ac93-5f42-48c8-85a0-848d53fa5056-tls-certs\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.682644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.682609    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp7jd\" (UniqueName: \"kubernetes.io/projected/ede9ac93-5f42-48c8-85a0-848d53fa5056-kube-api-access-kp7jd\") pod \"simulator-w0-68fbd1-kserve-67fd977c44-blckd\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") " pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:32.775523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:32.775460    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:33.144655 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:34:33.144628    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podede9ac93_5f42_48c8_85a0_848d53fa5056.slice/crio-d8940d8f01005d3b05c18c139dde9182de9c4ced48f86311758624d7b14de830 WatchSource:0}: Error finding container d8940d8f01005d3b05c18c139dde9182de9c4ced48f86311758624d7b14de830: Status 404 returned error can't find the container with id d8940d8f01005d3b05c18c139dde9182de9c4ced48f86311758624d7b14de830
Nov 03 22:34:33.147754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:33.147732    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"]
Nov 03 22:34:33.692966 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:33.692929    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" event={"ID":"ede9ac93-5f42-48c8-85a0-848d53fa5056","Type":"ContainerStarted","Data":"e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d"}
Nov 03 22:34:33.693366 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:33.692981    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" event={"ID":"ede9ac93-5f42-48c8-85a0-848d53fa5056","Type":"ContainerStarted","Data":"d8940d8f01005d3b05c18c139dde9182de9c4ced48f86311758624d7b14de830"}
Nov 03 22:34:33.776079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:33.776041    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:33.814812 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:33.814758    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podStartSLOduration=1.8147446820000002 podStartE2EDuration="1.814744682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:33.812602955 +0000 UTC m=+677.077302966" watchObservedRunningTime="2026-11-03 22:34:33.814744682 +0000 UTC m=+677.079444692"
Nov 03 22:34:34.970915 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:34.970882    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:39.485814 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:39.485775    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:40.083227 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:34:40.083196    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:34:41.280076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.280046    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"]
Nov 03 22:34:41.283476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.283456    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.338137 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.338109    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"]
Nov 03 22:34:41.386955 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.386929    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-6f1190\"/\"premium-w5-6f1190-kserve-self-signed-certs\""
Nov 03 22:34:41.407764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.407737    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-home\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.407939 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.407773    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6znsz\" (UniqueName: \"kubernetes.io/projected/9a193c2a-098b-49bd-9924-0b6b3fc70718-kube-api-access-6znsz\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.407939 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.407873    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-tmp-dir\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.407939 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.407903    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a193c2a-098b-49bd-9924-0b6b3fc70718-tls-certs\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.407939 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.407930    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-dshm\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.408077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.407968    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-model-cache\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.423443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.423403    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:41.509334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509308    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-home\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509338    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6znsz\" (UniqueName: \"kubernetes.io/projected/9a193c2a-098b-49bd-9924-0b6b3fc70718-kube-api-access-6znsz\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509457    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-tmp-dir\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509493    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a193c2a-098b-49bd-9924-0b6b3fc70718-tls-certs\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509524    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-dshm\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509843 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509663    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-model-cache\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509843 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509770    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-home\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509841    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-tmp-dir\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.509958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.509940    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-model-cache\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.511601 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.511581    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-dshm\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.511944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.511926    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a193c2a-098b-49bd-9924-0b6b3fc70718-tls-certs\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.572048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.571994    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6znsz\" (UniqueName: \"kubernetes.io/projected/9a193c2a-098b-49bd-9924-0b6b3fc70718-kube-api-access-6znsz\") pod \"premium-w5-6f1190-kserve-7974cbc8f6-b8lxt\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") " pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.595960 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.595936    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:41.987597 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:34:41.987568    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a193c2a_098b_49bd_9924_0b6b3fc70718.slice/crio-77dd732ab0ff579189428403859ee6afb57a74e155935f76dc5b9d21c0976b1a WatchSource:0}: Error finding container 77dd732ab0ff579189428403859ee6afb57a74e155935f76dc5b9d21c0976b1a: Status 404 returned error can't find the container with id 77dd732ab0ff579189428403859ee6afb57a74e155935f76dc5b9d21c0976b1a
Nov 03 22:34:41.990357 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:41.990337    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"]
Nov 03 22:34:42.752164 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:42.752130    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" event={"ID":"9a193c2a-098b-49bd-9924-0b6b3fc70718","Type":"ContainerStarted","Data":"95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c"}
Nov 03 22:34:42.752565 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:42.752176    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" event={"ID":"9a193c2a-098b-49bd-9924-0b6b3fc70718","Type":"ContainerStarted","Data":"77dd732ab0ff579189428403859ee6afb57a74e155935f76dc5b9d21c0976b1a"}
Nov 03 22:34:42.776568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:42.776544    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:42.789330 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:42.789306    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:42.874058 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:42.874008    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podStartSLOduration=1.8739956549999999 podStartE2EDuration="1.873995655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:42.871829723 +0000 UTC m=+686.136529734" watchObservedRunningTime="2026-11-03 22:34:42.873995655 +0000 UTC m=+686.138695698"
Nov 03 22:34:43.596573 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:43.596538    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:43.779118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:43.779091    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:34:43.974770 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:43.974724    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:46.209464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:46.209427    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:48.474297 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:48.474264    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:49.578158 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:49.578111    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:51.596740 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:51.596711    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:51.610061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:51.610031    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:51.823610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:51.823582    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:34:53.077048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:53.077016    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:34:57.585482 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:34:57.585372    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:00.535119 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.535088    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"]
Nov 03 22:35:00.539149 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.539132    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.593061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.593034    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"]
Nov 03 22:35:00.615078 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.615052    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/876c7542-5da1-42d2-8c62-57d41103f043-tls-certs\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.615211 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.615106    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-dshm\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.615211 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.615124    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-home\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.615211 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.615190    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph64m\" (UniqueName: \"kubernetes.io/projected/876c7542-5da1-42d2-8c62-57d41103f043-kube-api-access-ph64m\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.615370 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.615356    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-tmp-dir\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.615426 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.615414    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-model-cache\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.641832 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.641814    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-545b5d\"/\"simulator-w1-545b5d-kserve-self-signed-certs\""
Nov 03 22:35:00.641832 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.641826    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-545b5d\"/\"openshift-service-ca.crt\""
Nov 03 22:35:00.644602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.644585    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-545b5d\"/\"kube-root-ca.crt\""
Nov 03 22:35:00.644602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.644593    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-545b5d\"/\"default-dockercfg-s4kxl\""
Nov 03 22:35:00.688603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.688571    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:00.716667 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.716635    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-tmp-dir\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.716840 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.716692    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-model-cache\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.716840 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.716761    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/876c7542-5da1-42d2-8c62-57d41103f043-tls-certs\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.716840 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.716820    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-dshm\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.716997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.716844    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-home\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.716997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.716871    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph64m\" (UniqueName: \"kubernetes.io/projected/876c7542-5da1-42d2-8c62-57d41103f043-kube-api-access-ph64m\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.717138 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.717118    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-tmp-dir\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.717215 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.717181    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-model-cache\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.717270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.717213    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-home\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.719081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.719057    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-dshm\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.719242 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.719225    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/876c7542-5da1-42d2-8c62-57d41103f043-tls-certs\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.779716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.779691    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph64m\" (UniqueName: \"kubernetes.io/projected/876c7542-5da1-42d2-8c62-57d41103f043-kube-api-access-ph64m\") pod \"simulator-w1-545b5d-kserve-b689679d9-lqmv2\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") " pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:00.851963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:00.851892    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:01.220852 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:35:01.220820    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod876c7542_5da1_42d2_8c62_57d41103f043.slice/crio-cc8939a894749ca9c6bde8859039f41b1028a118f7f77284ef10f5947b14f002 WatchSource:0}: Error finding container cc8939a894749ca9c6bde8859039f41b1028a118f7f77284ef10f5947b14f002: Status 404 returned error can't find the container with id cc8939a894749ca9c6bde8859039f41b1028a118f7f77284ef10f5947b14f002
Nov 03 22:35:01.224148 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:01.224125    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"]
Nov 03 22:35:01.870019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:01.869982    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" event={"ID":"876c7542-5da1-42d2-8c62-57d41103f043","Type":"ContainerStarted","Data":"633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2"}
Nov 03 22:35:01.870434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:01.870030    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" event={"ID":"876c7542-5da1-42d2-8c62-57d41103f043","Type":"ContainerStarted","Data":"cc8939a894749ca9c6bde8859039f41b1028a118f7f77284ef10f5947b14f002"}
Nov 03 22:35:01.991168 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:01.991089    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podStartSLOduration=1.9910760440000002 podStartE2EDuration="1.991076044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:35:01.989522462 +0000 UTC m=+705.254222474" watchObservedRunningTime="2026-11-03 22:35:01.991076044 +0000 UTC m=+705.255776113"
Nov 03 22:35:02.081822 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:02.081786    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:02.854072 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:02.854044    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:06.579312 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:06.579269    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:10.084578 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:35:10.084540    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:35:10.854470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:10.854442    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:10.864788 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:10.864765    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:10.930675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:10.930654    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:35:11.095885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:11.095847    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:15.703717 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:15.703686    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:20.300043 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:20.300007    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:23.748218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:23.748176    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:24.787419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:24.787358    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:29.288929 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:29.288874    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:33.833532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:33.833476    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:35.217427 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:35.217338    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:38.370654 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:38.370579    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:40.083573 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:35:40.083529    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:35:42.866697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:42.866661    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:46.307099 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:46.307064    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:46.603351 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:46.603255    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:47.413044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:47.413010    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:51.898186 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:51.898140    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:53.103598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.103558    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"]
Nov 03 22:35:53.107501 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.107478    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.162131 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.162104    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"]
Nov 03 22:35:53.189500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.189474    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9p4t\" (UniqueName: \"kubernetes.io/projected/c73f4e32-0faa-48a1-8953-eca91366445c-kube-api-access-v9p4t\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.189646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.189534    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-home\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.189646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.189632    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-model-cache\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.189761 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.189700    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-tmp-dir\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.189835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.189817    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73f4e32-0faa-48a1-8953-eca91366445c-tls-certs\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.189895 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.189858    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-dshm\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.210527 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.210503    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-68fbd1\"/\"premium-w0-68fbd1-kserve-self-signed-certs\""
Nov 03 22:35:53.246339 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.246304    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:35:53.290616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.290579    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-tmp-dir\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.290810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.290638    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73f4e32-0faa-48a1-8953-eca91366445c-tls-certs\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.290810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.290685    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-dshm\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.290810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.290761    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9p4t\" (UniqueName: \"kubernetes.io/projected/c73f4e32-0faa-48a1-8953-eca91366445c-kube-api-access-v9p4t\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.290976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.290843    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-home\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.290976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.290907    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-model-cache\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.291463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.291149    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-tmp-dir\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.291463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.291199    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-model-cache\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.291463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.291410    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-home\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.293159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.293132    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-dshm\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.293251 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.293166    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73f4e32-0faa-48a1-8953-eca91366445c-tls-certs\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.358522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.358449    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9p4t\" (UniqueName: \"kubernetes.io/projected/c73f4e32-0faa-48a1-8953-eca91366445c-kube-api-access-v9p4t\") pod \"premium-w0-68fbd1-kserve-85ccbcc869-grk9j\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") " pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.419492 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.419459    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:53.815693 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:35:53.815665    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc73f4e32_0faa_48a1_8953_eca91366445c.slice/crio-e6703fc4c270be4f10603e74d7672799762bf5ddbe88de0c54149474cbe56d71 WatchSource:0}: Error finding container e6703fc4c270be4f10603e74d7672799762bf5ddbe88de0c54149474cbe56d71: Status 404 returned error can't find the container with id e6703fc4c270be4f10603e74d7672799762bf5ddbe88de0c54149474cbe56d71
Nov 03 22:35:53.818864 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:53.818689    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"]
Nov 03 22:35:54.167065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:54.166974    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" event={"ID":"c73f4e32-0faa-48a1-8953-eca91366445c","Type":"ContainerStarted","Data":"89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb"}
Nov 03 22:35:54.167065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:54.167019    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" event={"ID":"c73f4e32-0faa-48a1-8953-eca91366445c","Type":"ContainerStarted","Data":"e6703fc4c270be4f10603e74d7672799762bf5ddbe88de0c54149474cbe56d71"}
Nov 03 22:35:54.286838 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:54.286795    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podStartSLOduration=1.286783263 podStartE2EDuration="1.286783263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:35:54.28651246 +0000 UTC m=+757.551212472" watchObservedRunningTime="2026-11-03 22:35:54.286783263 +0000 UTC m=+757.551483299"
Nov 03 22:35:54.420509 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:54.420479    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:35:56.385756 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:35:56.385719    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:01.005711 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:01.005679    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:03.420490 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:03.420457    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:36:03.433364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:03.433336    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:36:04.240171 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:04.240145    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:36:05.478674 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:05.478639    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:09.981494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:09.981455    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:10.083735 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:36:10.083698    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:36:14.485694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:14.485657    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:15.126200 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:15.126162    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:18.988618 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:18.988578    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:23.474678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:23.474640    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:27.947370 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:27.947336    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:30.114261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.114226    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"]
Nov 03 22:36:30.117964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.117950    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.172701 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.172671    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"]
Nov 03 22:36:30.221066 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.221042    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-545b5d\"/\"premium-w1-545b5d-kserve-self-signed-certs\""
Nov 03 22:36:30.263430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.262506    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:30.299411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.299358    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tls-certs\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.299604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.299418    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-home\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.299604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.299529    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-dshm\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.299604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.299580    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-model-cache\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.299755 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.299679    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjdcp\" (UniqueName: \"kubernetes.io/projected/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-kube-api-access-wjdcp\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.299804 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.299782    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tmp-dir\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401181    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tls-certs\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401228    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-home\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401250    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-dshm\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401284    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-model-cache\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401334    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjdcp\" (UniqueName: \"kubernetes.io/projected/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-kube-api-access-wjdcp\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401381    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tmp-dir\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401803 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401640    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-home\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401803 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401743    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-model-cache\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.401903 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.401807    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tmp-dir\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.403663 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.403640    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-dshm\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.403968 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.403950    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tls-certs\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.475742 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.475670    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjdcp\" (UniqueName: \"kubernetes.io/projected/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-kube-api-access-wjdcp\") pod \"premium-w1-545b5d-kserve-5dc8ff84bd-4r692\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") " pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:30.729886 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:30.729855    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:31.102738 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:36:31.102709    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd520739d_3f3e_4ce3_add4_4dcf6a03a30a.slice/crio-d4aeb70398bde5d43732805aa3288fee55d45269f97d617f6f30c87d20c82198 WatchSource:0}: Error finding container d4aeb70398bde5d43732805aa3288fee55d45269f97d617f6f30c87d20c82198: Status 404 returned error can't find the container with id d4aeb70398bde5d43732805aa3288fee55d45269f97d617f6f30c87d20c82198
Nov 03 22:36:31.105416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:31.105371    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"]
Nov 03 22:36:31.387649 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:31.387563    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" event={"ID":"d520739d-3f3e-4ce3-add4-4dcf6a03a30a","Type":"ContainerStarted","Data":"efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77"}
Nov 03 22:36:31.387649 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:31.387606    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" event={"ID":"d520739d-3f3e-4ce3-add4-4dcf6a03a30a","Type":"ContainerStarted","Data":"d4aeb70398bde5d43732805aa3288fee55d45269f97d617f6f30c87d20c82198"}
Nov 03 22:36:31.508135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:31.508084    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podStartSLOduration=1.508069844 podStartE2EDuration="1.508069844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:36:31.506769658 +0000 UTC m=+794.771469669" watchObservedRunningTime="2026-11-03 22:36:31.508069844 +0000 UTC m=+794.772769855"
Nov 03 22:36:31.730754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:31.730721    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:32.424026 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:32.423991    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:36.920843 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:36.920809    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:40.083081 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:36:40.083041    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:36:40.730784 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:40.730749    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:40.742744 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:40.742716    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:41.393354 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:41.393301    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:41.460104 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:41.460073    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:36:46.082436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:46.082360    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:46.924377 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:46.924342    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:47.637657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:47.637622    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:47.904364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:47.904276    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:50.685718 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:50.685681    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:55.079525 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:55.079485    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:36:59.664424 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:36:59.664373    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:04.300773 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:04.300742    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:08.728231 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:08.726614    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:10.083725 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:37:10.083692    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:37:13.270430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:13.270366    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:17.714699 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:17.714662    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:22.202147 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:22.202101    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:26.780588 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:26.780537    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:31.304433 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:31.304400    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:35.881131 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:35.881090    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:40.084070 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:37:40.084031    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:37:40.384891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:40.384804    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:44.889615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:44.889580    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:47.317822 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:47.317783    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:47.566820 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:47.566791    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:49.030052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.030016    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"]
Nov 03 22:37:49.034275 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.034251    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.142806 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.142774    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"]
Nov 03 22:37:49.146500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146480    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146521    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146606    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146644    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146698 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146679    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8gcv\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-kube-api-access-z8gcv\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146750    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146807 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146791    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146831    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.146938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.146926    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.247858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.247815    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.247858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.247874    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.247970    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248032    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8gcv\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-kube-api-access-z8gcv\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248063    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248098    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248161    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248208    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248374 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248278    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248374 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248287    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248503 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248386    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248503 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248488    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248575    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.248719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.248701    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.250320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.250295    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.250569 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.250549    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.310456 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.310378    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.310584 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.310474    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8gcv\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-kube-api-access-z8gcv\") pod \"maas-default-gateway-openshift-default-59f957fdd7-l2qpx\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.348533 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.348489    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:49.382202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.382153    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:49.776407 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:37:49.776368    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47a8d9d_0538_459c_9bf4_0d46e30d8794.slice/crio-e783c126308e932dd982c9f1414299f730fde5cad82abea9ce68394913deb34c WatchSource:0}: Error finding container e783c126308e932dd982c9f1414299f730fde5cad82abea9ce68394913deb34c: Status 404 returned error can't find the container with id e783c126308e932dd982c9f1414299f730fde5cad82abea9ce68394913deb34c
Nov 03 22:37:49.778957 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.778914    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:37:49.779084 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.779002    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:37:49.779084 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.779041    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:37:49.779884 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.779855    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"]
Nov 03 22:37:49.865022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.864997    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" event={"ID":"f47a8d9d-0538-459c-9bf4-0d46e30d8794","Type":"ContainerStarted","Data":"ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6"}
Nov 03 22:37:49.865154 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.865054    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" event={"ID":"f47a8d9d-0538-459c-9bf4-0d46e30d8794","Type":"ContainerStarted","Data":"e783c126308e932dd982c9f1414299f730fde5cad82abea9ce68394913deb34c"}
Nov 03 22:37:49.989195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:49.989144    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podStartSLOduration=1.989121691 podStartE2EDuration="1.989121691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:37:49.987484412 +0000 UTC m=+873.252184426" watchObservedRunningTime="2026-11-03 22:37:49.989121691 +0000 UTC m=+873.253821758"
Nov 03 22:37:50.349160 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:50.349116    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:50.350917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:50.350893    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:50.351036 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:50.350941    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:51.369807 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:51.369765    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:51.370313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:51.369830    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:52.366431 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:52.365724    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:52.366431 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:52.365781    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:53.354762 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:53.354724    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:53.355414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:53.355357    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:53.799831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:53.799796    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:54.099016 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:54.098936    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:37:54.353785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:54.353695    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:54.354058 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:54.354028    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:55.356415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:55.352688    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:55.356415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:55.352755    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:56.349032 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:56.348993    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:56.349193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:56.349057    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:57.349294 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:57.349262    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 03 22:37:57.349707 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:57.349313    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 03 22:37:58.428710 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:58.428678    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:58.429168 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:58.428996    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:58.429920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:58.429897    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:37:58.868350 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:37:58.868269    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:03.391919 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:03.391864    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:07.900356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:07.900306    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:10.083751 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:38:10.083713    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:38:12.385263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:12.385230    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:16.526210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.526170    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"]
Nov 03 22:38:16.530222 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.530200    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.584612 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.584594    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-6f1190-openshift-default-dockercfg-fjmzm\""
Nov 03 22:38:16.605404 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605366    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605439    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605524 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605501    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605567    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605642    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605711 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605700    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857d62dc-6b25-4d48-b706-d995c5fc2fef-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605795    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5bpt\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-kube-api-access-d5bpt\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605864 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605853    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.605904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.605874    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.640776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.640747    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"]
Nov 03 22:38:16.706665 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.706635    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.706830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.706693    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.706830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.706721    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.706830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.706791    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.706997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.706832    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.706997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.706920    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857d62dc-6b25-4d48-b706-d995c5fc2fef-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707099 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707067    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5bpt\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-kube-api-access-d5bpt\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707155 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707112    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707223 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707150    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707223 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707183    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707223 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707216    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707569 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707545    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707602    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.707678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.707650    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857d62dc-6b25-4d48-b706-d995c5fc2fef-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.709030 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.709003    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.709333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.709314    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.768954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.768920    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.769434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.769380    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5bpt\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-kube-api-access-d5bpt\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:16.877977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.877899    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:16.899230 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.899203    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-6f1190-openshift-default-dockercfg-fjmzm\""
Nov 03 22:38:16.904465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:16.904443    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:17.342173 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:38:17.342137    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857d62dc_6b25_4d48_b706_d995c5fc2fef.slice/crio-3bf3a688a707d5841c3a672597b1e998f39eae873bec73f4b2de02acb275f82a WatchSource:0}: Error finding container 3bf3a688a707d5841c3a672597b1e998f39eae873bec73f4b2de02acb275f82a: Status 404 returned error can't find the container with id 3bf3a688a707d5841c3a672597b1e998f39eae873bec73f4b2de02acb275f82a
Nov 03 22:38:17.344364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:17.344340    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"]
Nov 03 22:38:17.344575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:17.344544    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:38:17.344650 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:17.344623    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:38:17.344714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:17.344666    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:38:18.070057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:18.070021    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" event={"ID":"857d62dc-6b25-4d48-b706-d995c5fc2fef","Type":"ContainerStarted","Data":"43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142"}
Nov 03 22:38:18.070567 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:18.070100    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" event={"ID":"857d62dc-6b25-4d48-b706-d995c5fc2fef","Type":"ContainerStarted","Data":"3bf3a688a707d5841c3a672597b1e998f39eae873bec73f4b2de02acb275f82a"}
Nov 03 22:38:18.195144 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:18.195091    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" podStartSLOduration=2.195078614 podStartE2EDuration="2.195078614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:38:18.193553391 +0000 UTC m=+901.458253402" watchObservedRunningTime="2026-11-03 22:38:18.195078614 +0000 UTC m=+901.459778625"
Nov 03 22:38:18.905274 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:18.905242    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:18.906718 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:18.906692    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 03 22:38:18.906835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:18.906746    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 03 22:38:19.905498 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:19.905463    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 03 22:38:19.905993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:19.905558    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 03 22:38:20.905596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:20.905561    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 03 22:38:20.906079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:20.905617    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 03 22:38:21.385766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:21.383280    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:21.640181 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:21.640100    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:21.905079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:21.905003    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 03 22:38:21.905079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:21.905058    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 03 22:38:22.905039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:22.905004    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 03 22:38:22.905429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:22.905058    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 03 22:38:24.109122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:24.109090    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:24.109607 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:24.109440    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:24.110288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:24.110266    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 22:38:25.838103 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:25.838063    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:30.289663 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:30.289627    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:32.204690 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.204654    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"]
Nov 03 22:38:32.208910 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.208888    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.263007 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.262986    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-545b5d-openshift-default-dockercfg-t5wb5\""
Nov 03 22:38:32.319623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.319592    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"]
Nov 03 22:38:32.324880 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.324855    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325095 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.324984    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54xp\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-kube-api-access-h54xp\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325095 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.325025    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325095 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.325058    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.325108    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.325169    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.325246    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.325293    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.325514 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.325324    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426466 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426437    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54xp\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-kube-api-access-h54xp\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426471    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426493    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426524    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426569    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426631    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426667    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426701    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426932 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426811    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.426932 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426861    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.427040 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.426973    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.427040 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.427029    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.427184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.427164    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.427439 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.427421    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.429044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.429020    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.429129 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.429106    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.488748 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.488694    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.488748 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.488697    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54xp\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-kube-api-access-h54xp\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.522321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.522297    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:32.945452 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:38:32.945424    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21fbf7f8_43dd_451b_973a_fdf8252cafa6.slice/crio-284a911414ea7d2e11758d9259fd397517d0f50496142d7347316d7169c6e8dd WatchSource:0}: Error finding container 284a911414ea7d2e11758d9259fd397517d0f50496142d7347316d7169c6e8dd: Status 404 returned error can't find the container with id 284a911414ea7d2e11758d9259fd397517d0f50496142d7347316d7169c6e8dd
Nov 03 22:38:32.947848 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.947811    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:38:32.947959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.947895    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:38:32.947959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.947938    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:38:32.948507 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:32.948487    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"]
Nov 03 22:38:33.160472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:33.160432    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" event={"ID":"21fbf7f8-43dd-451b-973a-fdf8252cafa6","Type":"ContainerStarted","Data":"a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717"}
Nov 03 22:38:33.160645 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:33.160502    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" event={"ID":"21fbf7f8-43dd-451b-973a-fdf8252cafa6","Type":"ContainerStarted","Data":"284a911414ea7d2e11758d9259fd397517d0f50496142d7347316d7169c6e8dd"}
Nov 03 22:38:33.284170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:33.284080    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podStartSLOduration=1.284065122 podStartE2EDuration="1.284065122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:38:33.283038015 +0000 UTC m=+916.547738027" watchObservedRunningTime="2026-11-03 22:38:33.284065122 +0000 UTC m=+916.548765198"
Nov 03 22:38:33.522801 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:33.522771    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:34.524755 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:34.524716    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:38:34.525142 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:34.524765    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:38:34.865463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:34.864922    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:35.522789 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:35.522757    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:38:35.522977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:35.522808    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:38:36.522903 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:36.522871    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:38:36.523291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:36.522922    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:38:37.523714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:37.523641    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:38:37.523714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:37.523695    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:38:38.523463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:38.523432    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:38:38.523633 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:38.523484    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:38:39.288799 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:39.288767    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:39.578287 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:39.578151    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:39.580458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:39.580430    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:39.583817 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:39.583507    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:38:40.083924 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:38:40.083884    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:38:43.143626 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:43.143586    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:43.788649 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:43.788613    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:46.269860 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:46.269818    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:46.870194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:46.870149    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:47.672185 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:47.672148    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:38:54.629591 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:38:54.629542    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:05.299596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:05.299556    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:09.237542 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.237497    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"]
Nov 03 22:39:09.241561 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.241541    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.275092 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.275064    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x4cb\" (UniqueName: \"kubernetes.io/projected/be81b953-ea9f-491b-be35-977a05fa2106-kube-api-access-4x4cb\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.275253 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.275228    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-tmp-dir\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.275343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.275327    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-home\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.275425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.275374    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-dshm\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.275491 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.275463    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-model-cache\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.275551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.275509    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be81b953-ea9f-491b-be35-977a05fa2106-tls-certs\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.297159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.297128    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"]
Nov 03 22:39:09.345651 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.345627    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-18c03a\"/\"kube-root-ca.crt\""
Nov 03 22:39:09.345801 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.345757    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-18c03a\"/\"distinct-w4-18c03a-kserve-self-signed-certs\""
Nov 03 22:39:09.347333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.347319    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-18c03a\"/\"openshift-service-ca.crt\""
Nov 03 22:39:09.347406 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.347339    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-18c03a\"/\"default-dockercfg-6xfnv\""
Nov 03 22:39:09.376341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376312    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-home\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.376523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376362    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-dshm\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.376523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376424    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-model-cache\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.376634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376544    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be81b953-ea9f-491b-be35-977a05fa2106-tls-certs\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.376634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376624    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x4cb\" (UniqueName: \"kubernetes.io/projected/be81b953-ea9f-491b-be35-977a05fa2106-kube-api-access-4x4cb\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.376747 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376722    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-home\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.376810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376745    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-tmp-dir\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.376861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376821    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-model-cache\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.377022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.376996    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-tmp-dir\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.378764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.378734    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-dshm\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.379008 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.378991    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be81b953-ea9f-491b-be35-977a05fa2106-tls-certs\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.416355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.416323    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:09.439324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.439295    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x4cb\" (UniqueName: \"kubernetes.io/projected/be81b953-ea9f-491b-be35-977a05fa2106-kube-api-access-4x4cb\") pod \"distinct-w4-18c03a-kserve-845b7c549f-cpknn\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") " pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.554259 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.554175    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:09.995790 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:39:09.995757    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe81b953_ea9f_491b_be35_977a05fa2106.slice/crio-f2297195291bccf18270a265678ae3389dbe5422c1b498abbc416b06e00edffb WatchSource:0}: Error finding container f2297195291bccf18270a265678ae3389dbe5422c1b498abbc416b06e00edffb: Status 404 returned error can't find the container with id f2297195291bccf18270a265678ae3389dbe5422c1b498abbc416b06e00edffb
Nov 03 22:39:09.999630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:09.999609    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"]
Nov 03 22:39:10.083819 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:39:10.083783    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:39:10.429024 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:10.428991    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" event={"ID":"be81b953-ea9f-491b-be35-977a05fa2106","Type":"ContainerStarted","Data":"8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7"}
Nov 03 22:39:10.429473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:10.429034    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" event={"ID":"be81b953-ea9f-491b-be35-977a05fa2106","Type":"ContainerStarted","Data":"f2297195291bccf18270a265678ae3389dbe5422c1b498abbc416b06e00edffb"}
Nov 03 22:39:10.549319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:10.549270    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podStartSLOduration=1.5492562589999999 podStartE2EDuration="1.549256259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:39:10.54899595 +0000 UTC m=+953.813695961" watchObservedRunningTime="2026-11-03 22:39:10.549256259 +0000 UTC m=+953.813956270"
Nov 03 22:39:10.555125 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:10.555103    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:13.030280 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:13.030246    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:18.781586 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:18.781538    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:19.554794 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:19.554756    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:19.568628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:19.568604    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:20.499550 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:20.499522    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:39:23.820593 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:23.820551    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:28.692286 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:28.692245    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:33.302767 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.302737    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"]
Nov 03 22:39:33.306764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.306748    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.361059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.361035    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-ee74cb-openshift-default-dockercfg-zszk2\""
Nov 03 22:39:33.374925 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.374897    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.374944    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375132 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.375092    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375169 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.375143    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxz74\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-kube-api-access-jxz74\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.375216    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375282 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.375262    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.375293    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375363 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.375328    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.375439 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.375422    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.417456 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.417429    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"]
Nov 03 22:39:33.477039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.476999    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477077    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxz74\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-kube-api-access-jxz74\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477114    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477143    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477183    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477226    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477281    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477314    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477369    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477481    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477500    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477561    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477836 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477816    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.477985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.477962    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.479539 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.479521    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.479958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.479941    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.539909 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.539882    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.540179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.540154    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxz74\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-kube-api-access-jxz74\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.620235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.620141    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:33.627813 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:33.627788    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:34.058187 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:39:34.058153    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12f2950_90f4_4c21_97ab_ef5a5541d9ab.slice/crio-ebace35289a3a9cf35bd021a9b91c3346eae04c53df0e468429d83d3068882d1 WatchSource:0}: Error finding container ebace35289a3a9cf35bd021a9b91c3346eae04c53df0e468429d83d3068882d1: Status 404 returned error can't find the container with id ebace35289a3a9cf35bd021a9b91c3346eae04c53df0e468429d83d3068882d1
Nov 03 22:39:34.060818 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.060783    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:39:34.060907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.060848    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:39:34.060907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.060876    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:39:34.061302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.061238    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"]
Nov 03 22:39:34.579505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.579461    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" event={"ID":"f12f2950-90f4-4c21-97ab-ef5a5541d9ab","Type":"ContainerStarted","Data":"ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218"}
Nov 03 22:39:34.579938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.579584    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" event={"ID":"f12f2950-90f4-4c21-97ab-ef5a5541d9ab","Type":"ContainerStarted","Data":"ebace35289a3a9cf35bd021a9b91c3346eae04c53df0e468429d83d3068882d1"}
Nov 03 22:39:34.620347 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.620319    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:34.704257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:34.704203    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podStartSLOduration=1.704188406 podStartE2EDuration="1.704188406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:39:34.702303607 +0000 UTC m=+977.967003618" watchObservedRunningTime="2026-11-03 22:39:34.704188406 +0000 UTC m=+977.968888416"
Nov 03 22:39:35.622443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:35.622383    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:35.622848 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:35.622466    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:36.620568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:36.620531    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:36.620768 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:36.620589    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:37.621457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:37.621423    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:37.621853 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:37.621481    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:38.563974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:38.563938    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:38.621000 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:38.620966    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:38.621166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:38.621014    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:38.773438 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:38.773405    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:39.620893 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:39.620861    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:39.621073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:39.620920    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:40.090883 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:39:40.090849    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:39:40.620875 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:40.620843    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:40.621077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:40.620906    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:41.625291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:41.625263    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:41.625802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:41.625679    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:41.627510 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:41.627488    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:39:43.622699 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:43.622641    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:48.038359 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.038323    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:48.140552 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.140513    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"]
Nov 03 22:39:48.144419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.144380    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.179251 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.179224    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tls-certs\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.179411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.179256    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tmp-dir\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.179411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.179361    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-home\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.179548 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.179485    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-model-cache\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.179632 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.179607    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-kube-api-access-zwkrc\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.179723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.179706    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-dshm\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.255874 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.255846    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-68fbd1\"/\"distinct-2-w0-68fbd1-kserve-self-signed-certs\""
Nov 03 22:39:48.258935 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.258914    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"]
Nov 03 22:39:48.280752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.280697    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-home\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.281034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.281013    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-model-cache\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.281265 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.281235    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-kube-api-access-zwkrc\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.281545 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.281513    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-dshm\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.281774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.281753    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tls-certs\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.286010 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.282502    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-model-cache\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.286010 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.282638    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tmp-dir\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.286010 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.282750    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-home\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.286010 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.283055    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tmp-dir\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.288776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.288676    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-dshm\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.297290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.292546    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tls-certs\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.306170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.304017    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:48.351480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.351452    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-kube-api-access-zwkrc\") pod \"distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") " pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.457308 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.457278    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:48.549113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.549026    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:48.829682 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:39:48.829629    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59552c85_6ea7_42f0_9b94_eb639f8f0dfd.slice/crio-11c6cfa60f9ebca47433c5ff6ae86a6c46bc9d6301a4e42a36fe5cbf8b81f8e3 WatchSource:0}: Error finding container 11c6cfa60f9ebca47433c5ff6ae86a6c46bc9d6301a4e42a36fe5cbf8b81f8e3: Status 404 returned error can't find the container with id 11c6cfa60f9ebca47433c5ff6ae86a6c46bc9d6301a4e42a36fe5cbf8b81f8e3
Nov 03 22:39:48.833268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:48.833242    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"]
Nov 03 22:39:49.680014 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:49.679977    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" event={"ID":"59552c85-6ea7-42f0-9b94-eb639f8f0dfd","Type":"ContainerStarted","Data":"ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192"}
Nov 03 22:39:49.680424 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:49.680019    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" event={"ID":"59552c85-6ea7-42f0-9b94-eb639f8f0dfd","Type":"ContainerStarted","Data":"11c6cfa60f9ebca47433c5ff6ae86a6c46bc9d6301a4e42a36fe5cbf8b81f8e3"}
Nov 03 22:39:49.836625 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:49.836573    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podStartSLOduration=1.8365597519999999 podStartE2EDuration="1.836559752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:39:49.834414495 +0000 UTC m=+993.099114498" watchObservedRunningTime="2026-11-03 22:39:49.836559752 +0000 UTC m=+993.101259792"
Nov 03 22:39:50.458349 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:50.458316    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:53.475865 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:53.475821    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:58.014367 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:58.014329    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:58.408344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:58.407849    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:39:58.458376 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:58.458348    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:58.472031 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:58.472007    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:39:58.756462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:39:58.756428    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:40:03.449139 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:03.449091    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:08.340034 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:08.339935    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:10.084517 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:40:10.084482    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:40:13.337313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:13.337282    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:18.237917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:18.237882    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:23.203504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:23.203453    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:28.132327 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:28.132285    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:30.868173 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:30.868146    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"]
Nov 03 22:40:30.872244 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:30.872228    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:30.927994 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:30.927965    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"]
Nov 03 22:40:30.975364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:30.975335    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-961054\"/\"kube-root-ca.crt\""
Nov 03 22:40:30.975547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:30.975486    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-961054\"/\"openshift-service-ca.crt\""
Nov 03 22:40:30.975617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:30.975601    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-961054\"/\"default-dockercfg-9ssgv\""
Nov 03 22:40:30.977847 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:30.977826    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-961054\"/\"distinct-2-w3-961054-kserve-self-signed-certs\""
Nov 03 22:40:31.018365 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.018329    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-model-cache\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.018713 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.018692    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-home\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.018987 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.018971    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4783432e-dd9f-4416-b435-c6960145a39c-tls-certs\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.019239 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.019222    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-dshm\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.019450 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.019433    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-tmp-dir\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.020821 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.020754    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgqf4\" (UniqueName: \"kubernetes.io/projected/4783432e-dd9f-4416-b435-c6960145a39c-kube-api-access-hgqf4\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.022765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.022372    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.123232    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgqf4\" (UniqueName: \"kubernetes.io/projected/4783432e-dd9f-4416-b435-c6960145a39c-kube-api-access-hgqf4\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.123377    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-model-cache\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.123458    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-home\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.123533    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4783432e-dd9f-4416-b435-c6960145a39c-tls-certs\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.123635    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-dshm\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.123695    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-tmp-dir\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.124223    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-tmp-dir\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.124859    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-model-cache\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.127734    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4783432e-dd9f-4416-b435-c6960145a39c-tls-certs\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.128540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.128148    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-home\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.134274 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.134250    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-dshm\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.186719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.186692    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgqf4\" (UniqueName: \"kubernetes.io/projected/4783432e-dd9f-4416-b435-c6960145a39c-kube-api-access-hgqf4\") pod \"distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") " pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.485861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.485832    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:31.864616 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:40:31.864580    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4783432e_dd9f_4416_b435_c6960145a39c.slice/crio-b925a6bc156fb27cb584cabc3b2fbb659db7af616862700d3faa66a56b5c357a WatchSource:0}: Error finding container b925a6bc156fb27cb584cabc3b2fbb659db7af616862700d3faa66a56b5c357a: Status 404 returned error can't find the container with id b925a6bc156fb27cb584cabc3b2fbb659db7af616862700d3faa66a56b5c357a
Nov 03 22:40:31.867450 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.867430    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"]
Nov 03 22:40:31.958687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.958651    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" event={"ID":"4783432e-dd9f-4416-b435-c6960145a39c","Type":"ContainerStarted","Data":"464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0"}
Nov 03 22:40:31.959014 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:31.958697    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" event={"ID":"4783432e-dd9f-4416-b435-c6960145a39c","Type":"ContainerStarted","Data":"b925a6bc156fb27cb584cabc3b2fbb659db7af616862700d3faa66a56b5c357a"}
Nov 03 22:40:32.081016 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:32.080964    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podStartSLOduration=2.080949184 podStartE2EDuration="2.080949184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:40:32.078495119 +0000 UTC m=+1035.343195131" watchObservedRunningTime="2026-11-03 22:40:32.080949184 +0000 UTC m=+1035.345649194"
Nov 03 22:40:32.486052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:32.486023    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:33.033075 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:33.032897    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:37.924081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:37.924041    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:40.083580 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:40:40.083545    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:40:41.486649 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:41.486618    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:41.499291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:41.499257    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:42.031159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:42.031134    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:40:42.827180 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:42.827142    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:47.767681 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:47.767648    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:48.387368 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:48.387328    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:52.630202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:52.630156    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:57.553712 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:57.553659    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:58.339582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:58.339530    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:40:59.615565 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:40:59.615526    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:02.422645 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:02.422605    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:04.347258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:04.347211    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:06.752490 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:06.752433    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:07.354430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:07.354398    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:10.083386 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:41:10.083351    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:41:12.253746 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:12.253709    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:14.947506 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.947474    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"]
Nov 03 22:41:14.951660 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.951638    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:14.993320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.993292    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-model-cache\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:14.993487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.993329    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f2fp\" (UniqueName: \"kubernetes.io/projected/fe0cad46-96b3-44fb-81a4-9415d4db7046-kube-api-access-5f2fp\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:14.993487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.993353    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-dshm\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:14.993575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.993519    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0cad46-96b3-44fb-81a4-9415d4db7046-tls-certs\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:14.993634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.993619    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-home\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:14.993683 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:14.993645    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-tmp-dir\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.010156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.010127    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"]
Nov 03 22:41:15.056429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.056379    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-6f1190\"/\"distinct-2-w5-6f1190-kserve-self-signed-certs\""
Nov 03 22:41:15.087509 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.087467    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:15.098718 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.098654    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f2fp\" (UniqueName: \"kubernetes.io/projected/fe0cad46-96b3-44fb-81a4-9415d4db7046-kube-api-access-5f2fp\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.098718 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.098725    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-dshm\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.098993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.098808    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0cad46-96b3-44fb-81a4-9415d4db7046-tls-certs\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.098993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.098910    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-home\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.098993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.098944    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-tmp-dir\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.099257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.099236    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-model-cache\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.099751 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.099730    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-model-cache\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.100854 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.100828    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-home\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.101078 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.101054    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-tmp-dir\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.104002 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.103972    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-dshm\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.108253 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.108230    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0cad46-96b3-44fb-81a4-9415d4db7046-tls-certs\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.169171 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.169136    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f2fp\" (UniqueName: \"kubernetes.io/projected/fe0cad46-96b3-44fb-81a4-9415d4db7046-kube-api-access-5f2fp\") pod \"distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") " pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.264960 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.264888    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:15.656734 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:41:15.656706    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe0cad46_96b3_44fb_81a4_9415d4db7046.slice/crio-eff0de5d6834feef7bec578c28d93ad8d366f47fab80142c2d41becbffc65841 WatchSource:0}: Error finding container eff0de5d6834feef7bec578c28d93ad8d366f47fab80142c2d41becbffc65841: Status 404 returned error can't find the container with id eff0de5d6834feef7bec578c28d93ad8d366f47fab80142c2d41becbffc65841
Nov 03 22:41:15.659475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:15.659453    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"]
Nov 03 22:41:16.235157 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:16.235121    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" event={"ID":"fe0cad46-96b3-44fb-81a4-9415d4db7046","Type":"ContainerStarted","Data":"478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde"}
Nov 03 22:41:16.235538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:16.235169    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" event={"ID":"fe0cad46-96b3-44fb-81a4-9415d4db7046","Type":"ContainerStarted","Data":"eff0de5d6834feef7bec578c28d93ad8d366f47fab80142c2d41becbffc65841"}
Nov 03 22:41:16.265279 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:16.265247    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:16.358418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:16.358357    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podStartSLOduration=2.358342978 podStartE2EDuration="2.358342978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:16.355652116 +0000 UTC m=+1079.620352127" watchObservedRunningTime="2026-11-03 22:41:16.358342978 +0000 UTC m=+1079.623042989"
Nov 03 22:41:17.134072 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:17.134042    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:18.808071 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.808042    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"]
Nov 03 22:41:18.812216 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.812201    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.853714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.853674    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.853872 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.853738    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.853872 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.853776    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.853872 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.853818    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.854042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.853914    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.854042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.853962    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.854042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.854001    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsffg\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-kube-api-access-jsffg\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.854042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.854036    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.854242 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.854087    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.871018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.870990    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"]
Nov 03 22:41:18.955904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.955874    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956040    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956099    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956150 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956134    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956166    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956242    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956289    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956368 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956324    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsffg\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-kube-api-access-jsffg\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956368 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956359    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956371    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956463    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956686 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956663    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.956911 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956890    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.957000 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.956980    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.958212 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.958188    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:18.958483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:18.958466    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:19.018847 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:19.018818    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:19.018963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:19.018924    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsffg\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-kube-api-access-jsffg\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:19.125899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:19.125829    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:19.551594 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:41:19.551546    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c8f4d07_e09c_40dd_8370_a15b6dc5c26d.slice/crio-512bd3959b070e21e76dde6d151036ae06a8702096f81d2538e584350a45b83c WatchSource:0}: Error finding container 512bd3959b070e21e76dde6d151036ae06a8702096f81d2538e584350a45b83c: Status 404 returned error can't find the container with id 512bd3959b070e21e76dde6d151036ae06a8702096f81d2538e584350a45b83c
Nov 03 22:41:19.553780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:19.553755    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"]
Nov 03 22:41:19.554232 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:19.554204    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:41:19.554290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:19.554271    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:41:19.554342 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:19.554300    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:41:20.287691 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:20.287654    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" event={"ID":"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d","Type":"ContainerStarted","Data":"df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf"}
Nov 03 22:41:20.288273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:20.288247    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" event={"ID":"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d","Type":"ContainerStarted","Data":"512bd3959b070e21e76dde6d151036ae06a8702096f81d2538e584350a45b83c"}
Nov 03 22:41:20.419583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:20.419523    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podStartSLOduration=2.419509335 podStartE2EDuration="2.419509335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:20.417717397 +0000 UTC m=+1083.682417407" watchObservedRunningTime="2026-11-03 22:41:20.419509335 +0000 UTC m=+1083.684209345"
Nov 03 22:41:21.126463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:21.126431    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:21.127831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:21.127809    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 03 22:41:21.127946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:21.127862    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 03 22:41:22.126593 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:22.126557    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 03 22:41:22.127005 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:22.126634    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 03 22:41:22.278473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:22.277055    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:23.126608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:23.126573    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 03 22:41:23.127011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:23.126630    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 03 22:41:24.126580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:24.126547    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 03 22:41:24.126742 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:24.126593    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 03 22:41:24.182563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:24.182527    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:25.126412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:25.126363    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 03 22:41:25.126597 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:25.126449    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 03 22:41:25.265664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:25.265634    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:25.284011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:25.283984    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:25.382202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:25.382125    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:41:26.126977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:26.126942    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 03 22:41:26.127150 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:26.127017    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 03 22:41:27.423192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:27.423157    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:28.001033 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:28.001003    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:28.001319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:28.001295    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:28.002098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:28.002078    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:41:31.236209 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.236179    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"]
Nov 03 22:41:31.240008 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.239989    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.295992 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.295963    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"]
Nov 03 22:41:31.342756 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.342731    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-5e72ad\"/\"openshift-service-ca.crt\""
Nov 03 22:41:31.342907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.342761    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-5e72ad\"/\"kube-root-ca.crt\""
Nov 03 22:41:31.345514 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.345493    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-5e72ad\"/\"default-dockercfg-pkpfl\""
Nov 03 22:41:31.345611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.345527    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-5e72ad\"/\"distinct-2-w6-5e72ad-kserve-self-signed-certs\""
Nov 03 22:41:31.363356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.363334    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-home\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.363474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.363358    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-dshm\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.363474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.363375    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4hqg\" (UniqueName: \"kubernetes.io/projected/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-kube-api-access-m4hqg\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.363572 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.363559    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tls-certs\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.363668 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.363655    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-model-cache\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.363809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.363792    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tmp-dir\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.390144 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.390110    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:31.464509 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464477    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tmp-dir\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.464509 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464515    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-home\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.464770 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464616    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-dshm\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.464770 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464645    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4hqg\" (UniqueName: \"kubernetes.io/projected/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-kube-api-access-m4hqg\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.464891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464786    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tls-certs\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.464891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464872    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-home\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.465001 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464925    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-model-cache\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.465001 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.464927    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tmp-dir\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.465186 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.465164    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-model-cache\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.466892 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.466863    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-dshm\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.467135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.467118    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tls-certs\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.527259 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.527197    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4hqg\" (UniqueName: \"kubernetes.io/projected/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-kube-api-access-m4hqg\") pod \"distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") " pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.553168 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.553145    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:31.935413 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:41:31.935373    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d28165a_2bf2_44b9_be71_d3cc7ed742e5.slice/crio-956bf034f62ce587af4d8a8bcda83105c29655aaf856231cba9601248fd31704 WatchSource:0}: Error finding container 956bf034f62ce587af4d8a8bcda83105c29655aaf856231cba9601248fd31704: Status 404 returned error can't find the container with id 956bf034f62ce587af4d8a8bcda83105c29655aaf856231cba9601248fd31704
Nov 03 22:41:31.939571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:31.939154    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"]
Nov 03 22:41:32.423423 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:32.422501    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:32.438185 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:32.438144    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" event={"ID":"2d28165a-2bf2-44b9-be71-d3cc7ed742e5","Type":"ContainerStarted","Data":"c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3"}
Nov 03 22:41:32.438357 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:32.438199    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" event={"ID":"2d28165a-2bf2-44b9-be71-d3cc7ed742e5","Type":"ContainerStarted","Data":"956bf034f62ce587af4d8a8bcda83105c29655aaf856231cba9601248fd31704"}
Nov 03 22:41:32.554022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:32.553992    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:32.558317 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:32.558274    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podStartSLOduration=1.5582608420000001 podStartE2EDuration="1.558260842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:32.557347381 +0000 UTC m=+1095.822047395" watchObservedRunningTime="2026-11-03 22:41:32.558260842 +0000 UTC m=+1095.822960852"
Nov 03 22:41:37.243203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:37.243112    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:40.084556 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:41:40.084519    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:41:40.989478 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:40.989442    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"]
Nov 03 22:41:40.993603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:40.993582    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.049610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.049582    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"]
Nov 03 22:41:41.091765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.091734    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-home\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.092183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.091772    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tls-certs\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.092183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.091875    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-model-cache\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.092183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.091924    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tmp-dir\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.092183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.092000    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v226s\" (UniqueName: \"kubernetes.io/projected/c87f5c2c-0676-46ec-8512-8140bcaf92a8-kube-api-access-v226s\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.092183 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.092057    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-dshm\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.096724 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.096703    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-545b5d\"/\"distinct-2-w1-545b5d-kserve-self-signed-certs\""
Nov 03 22:41:41.126329 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.126299    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:41.193675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.193642    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-home\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.193878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.193688    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tls-certs\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.193878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.193746    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-model-cache\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.193878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.193772    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tmp-dir\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.193878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.193812    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v226s\" (UniqueName: \"kubernetes.io/projected/c87f5c2c-0676-46ec-8512-8140bcaf92a8-kube-api-access-v226s\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.193878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.193861    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-dshm\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.194132 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.194072    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-home\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.194132 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.194120    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-model-cache\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.194255 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.194234    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tmp-dir\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.196188 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.196162    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-dshm\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.196429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.196413    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tls-certs\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.259962 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.259907    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v226s\" (UniqueName: \"kubernetes.io/projected/c87f5c2c-0676-46ec-8512-8140bcaf92a8-kube-api-access-v226s\") pod \"distinct-2-w1-545b5d-kserve-667b564f6-gsj8t\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") " pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.306127 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.306101    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:41.553494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.553416    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:41.565753 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.565730    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:41.728799 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:41:41.728775    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc87f5c2c_0676_46ec_8512_8140bcaf92a8.slice/crio-30777f6ad881356af3a3ec3a556624bb4319a661e394f6d6674b74ad49454d18 WatchSource:0}: Error finding container 30777f6ad881356af3a3ec3a556624bb4319a661e394f6d6674b74ad49454d18: Status 404 returned error can't find the container with id 30777f6ad881356af3a3ec3a556624bb4319a661e394f6d6674b74ad49454d18
Nov 03 22:41:41.741094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:41.741069    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"]
Nov 03 22:41:42.222619 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:42.222582    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:42.512119 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:42.512021    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" event={"ID":"c87f5c2c-0676-46ec-8512-8140bcaf92a8","Type":"ContainerStarted","Data":"9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba"}
Nov 03 22:41:42.512119 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:42.512076    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" event={"ID":"c87f5c2c-0676-46ec-8512-8140bcaf92a8","Type":"ContainerStarted","Data":"30777f6ad881356af3a3ec3a556624bb4319a661e394f6d6674b74ad49454d18"}
Nov 03 22:41:42.524810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:42.524785    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:41:42.634698 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:42.634652    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podStartSLOduration=2.6346386280000003 podStartE2EDuration="2.634638628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:42.631521445 +0000 UTC m=+1105.896221470" watchObservedRunningTime="2026-11-03 22:41:42.634638628 +0000 UTC m=+1105.899338638"
Nov 03 22:41:43.306334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:43.306303    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:47.118385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:47.118347    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:48.740525 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:48.740492    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:49.369262 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.369234    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"]
Nov 03 22:41:49.373669 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.373603    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.484773 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.484742    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"]
Nov 03 22:41:49.494014 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.493962    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494175 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494111    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bbrz\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-kube-api-access-6bbrz\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494175 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494155    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6ce9553-9209-4c0d-b8cf-484612f07209-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494285 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494192    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494357 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494334    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494470    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494554 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494503    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494610    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.494723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.494709    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596423    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596479    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bbrz\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-kube-api-access-6bbrz\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596505    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6ce9553-9209-4c0d-b8cf-484612f07209-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596627 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596530    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596743    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596815    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596894 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596870    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596902    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.596997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.596986    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.597088 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.597070    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.597213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.597188    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.597278 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.597231    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.597338 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.597288    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6ce9553-9209-4c0d-b8cf-484612f07209-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.597338 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.597309    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.599032 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.599014    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.599129 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.599086    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.664553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.664489    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.664774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.664750    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bbrz\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-kube-api-access-6bbrz\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:49.688583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:49.688555    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:50.115246 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:41:50.115219    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ce9553_9209_4c0d_b8cf_484612f07209.slice/crio-bafd99c6905f0217060261e8b2e0be664143ba06912ca520a53e8d72f54d7dca WatchSource:0}: Error finding container bafd99c6905f0217060261e8b2e0be664143ba06912ca520a53e8d72f54d7dca: Status 404 returned error can't find the container with id bafd99c6905f0217060261e8b2e0be664143ba06912ca520a53e8d72f54d7dca
Nov 03 22:41:50.117818 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.117787    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:41:50.117897 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.117854    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:41:50.117897 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.117886    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:41:50.119427 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.119405    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"]
Nov 03 22:41:50.565807 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.565766    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" event={"ID":"f6ce9553-9209-4c0d-b8cf-484612f07209","Type":"ContainerStarted","Data":"df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884"}
Nov 03 22:41:50.565982 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.565822    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" event={"ID":"f6ce9553-9209-4c0d-b8cf-484612f07209","Type":"ContainerStarted","Data":"bafd99c6905f0217060261e8b2e0be664143ba06912ca520a53e8d72f54d7dca"}
Nov 03 22:41:50.688993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.688844    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:50.695472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:50.695427    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podStartSLOduration=1.695412303 podStartE2EDuration="1.695412303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:50.693714617 +0000 UTC m=+1113.958414627" watchObservedRunningTime="2026-11-03 22:41:50.695412303 +0000 UTC m=+1113.960112307"
Nov 03 22:41:51.307208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:51.307175    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:51.319508 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:51.319484    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:51.704474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:51.702014    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 03 22:41:51.704474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:51.702112    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 03 22:41:51.820415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:51.819780    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:41:52.090484 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:52.090327    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:52.698415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:52.690750    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 03 22:41:52.698415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:52.690814    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 03 22:41:53.303416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.299508    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:53.385414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.375699    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:53.466423 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.464466    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:41:53.466423 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.465445    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:53.506417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.502498    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:53.527413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.526469    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:53.659471 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.658771    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:53.692415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.691724    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 03 22:41:53.692415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.691780    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 03 22:41:53.771415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.770560    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:53.782415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:53.780464    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:54.112761 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:54.025487    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:41:54.689267 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:54.689233    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 03 22:41:54.689496 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:54.689284    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 03 22:41:55.688942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:55.688909    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 03 22:41:55.689355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:55.688959    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 03 22:41:56.689829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:56.689797    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 03 22:41:56.690237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:56.689844    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 03 22:41:57.242708 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:57.242659    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:41:57.777526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:57.777492    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:57.777981 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:57.777848    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:57.778618 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:57.778599    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 22:41:58.726506 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:41:58.726473    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:02.135650 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:02.135606    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:07.058774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:07.058731    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:10.083905 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:42:10.083871    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:42:11.968523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:11.968492    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:17.185194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:17.185161    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:18.284304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.284267    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"]
Nov 03 22:42:18.288298 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.288283    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.376725 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.376689    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.376944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.376782    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.376944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.376850    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.376944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.376911    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02cc5b72-9510-4de2-be26-c2b2142af282-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.376944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.376933    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02cc5b72-9510-4de2-be26-c2b2142af282-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.377217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.376973    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.377217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.377104    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spq2c\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-kube-api-access-spq2c\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.377217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.377139    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.377217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.377159    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.397958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.397932    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"]
Nov 03 22:42:18.478173 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478137    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478206    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478245    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478282    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02cc5b72-9510-4de2-be26-c2b2142af282-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478408    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02cc5b72-9510-4de2-be26-c2b2142af282-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478457    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478592    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478610    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478602    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spq2c\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-kube-api-access-spq2c\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478671    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.478733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478700    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.479006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.478980    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.479145 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.479120    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02cc5b72-9510-4de2-be26-c2b2142af282-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.479271 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.479249    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.481135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.481106    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02cc5b72-9510-4de2-be26-c2b2142af282-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.481278 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.481257    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.542664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.542602    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.546451 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.546430    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spq2c\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-kube-api-access-spq2c\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.585782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.585751    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"]
Nov 03 22:42:18.590811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.590793    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.602226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.602204    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:18.647656 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.647629    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"]
Nov 03 22:42:18.681618 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.681498    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istiod-ca-cert\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.681618 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.681598    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-envoy\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.681818 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.681630    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-podinfo\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.681818 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.681693    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-token\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.681818 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.681724    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-certs\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.681818 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.681803    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.682033 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.681834    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-credential-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.683257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.682112    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thd5l\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-kube-api-access-thd5l\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.683257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.682166    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-data\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.783779 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.783747    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thd5l\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-kube-api-access-thd5l\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.783779 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.783783    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-data\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.783997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.783827    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istiod-ca-cert\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.783997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.783879    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-envoy\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.783997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.783983    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-podinfo\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784070    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-token\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784105    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-certs\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784252 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784188    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784252 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784220    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-credential-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784349 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784308    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-data\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784558 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784528    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-certs\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784567    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784654    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-credential-socket\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.784703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.784662    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istiod-ca-cert\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.786244 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.786227    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-envoy\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.786372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.786357    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-podinfo\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.848288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.848225    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-token\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.848734 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.848714    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thd5l\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-kube-api-access-thd5l\") pod \"e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") " pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:18.904063 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:18.904032    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:19.034440 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:42:19.034377    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02cc5b72_9510_4de2_be26_c2b2142af282.slice/crio-4e01858b35113e6019e9a514cf757bf1fd6209cb555f743cd44a764c307859c1 WatchSource:0}: Error finding container 4e01858b35113e6019e9a514cf757bf1fd6209cb555f743cd44a764c307859c1: Status 404 returned error can't find the container with id 4e01858b35113e6019e9a514cf757bf1fd6209cb555f743cd44a764c307859c1
Nov 03 22:42:19.037554 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.037521    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.037640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.037586    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.037640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.037618    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.038572 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.038548    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"]
Nov 03 22:42:19.237295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.237265    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"]
Nov 03 22:42:19.241491 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.241467    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.284418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.284362    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.284878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.284848    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.284986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.284908    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.290708 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.290684    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"]
Nov 03 22:42:19.293225 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293201    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.293418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293376    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.293575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293561    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.293738 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293714    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.293902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293750    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.293902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293780    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.293902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293863    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.293902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293889    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037e749e-a34d-4610-a3da-9eaef32a46ab-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.294886 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.293921    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dgmx\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-kube-api-access-6dgmx\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.328359 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.328333    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"]
Nov 03 22:42:19.395855 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.395821    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.395869    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.395899    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.395993    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396196 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396021    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037e749e-a34d-4610-a3da-9eaef32a46ab-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396196 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396059    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dgmx\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-kube-api-access-6dgmx\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396208    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-data\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396229    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396275    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396342    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396760    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396884    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037e749e-a34d-4610-a3da-9eaef32a46ab-istiod-ca-cert\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.396904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.396893    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-certs\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.398591 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.398564    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-credential-socket\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.399299 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.399273    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-podinfo\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.399468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.399332    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-envoy\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.459385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.459350    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-token\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.459581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.459559    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dgmx\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-kube-api-access-6dgmx\") pod \"e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") " pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.556132 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.556051    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:19.855030 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.854934    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" event={"ID":"8679ecba-3f23-4fc3-8426-d7a29c6bde30","Type":"ContainerStarted","Data":"bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90"}
Nov 03 22:42:19.855337 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.855310    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" event={"ID":"8679ecba-3f23-4fc3-8426-d7a29c6bde30","Type":"ContainerStarted","Data":"71b416e4bb5c948a43cb2b4ebeb8c38d55f3c40db744b188db0b5375f8a00d58"}
Nov 03 22:42:19.858943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.858908    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" event={"ID":"02cc5b72-9510-4de2-be26-c2b2142af282","Type":"ContainerStarted","Data":"73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7"}
Nov 03 22:42:19.859156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.859135    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" event={"ID":"02cc5b72-9510-4de2-be26-c2b2142af282","Type":"ContainerStarted","Data":"4e01858b35113e6019e9a514cf757bf1fd6209cb555f743cd44a764c307859c1"}
Nov 03 22:42:19.904278 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.904241    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:19.968591 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.968564    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"]
Nov 03 22:42:19.971876 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:42:19.971848    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037e749e_a34d_4610_a3da_9eaef32a46ab.slice/crio-b8fa7a8dfb655857e962df9bbbb4e24d2f9e1d0cd24926c69869d5f3c57784c9 WatchSource:0}: Error finding container b8fa7a8dfb655857e962df9bbbb4e24d2f9e1d0cd24926c69869d5f3c57784c9: Status 404 returned error can't find the container with id b8fa7a8dfb655857e962df9bbbb4e24d2f9e1d0cd24926c69869d5f3c57784c9
Nov 03 22:42:19.975956 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.975915    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.976176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.976144    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.976282 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.976201    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:42:19.981938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:19.981898    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podStartSLOduration=1.98188666 podStartE2EDuration="1.98188666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:42:19.978892592 +0000 UTC m=+1143.243592622" watchObservedRunningTime="2026-11-03 22:42:19.98188666 +0000 UTC m=+1143.246586670"
Nov 03 22:42:20.102574 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.102520    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podStartSLOduration=2.102503098 podStartE2EDuration="2.102503098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:42:20.102089911 +0000 UTC m=+1143.366789921" watchObservedRunningTime="2026-11-03 22:42:20.102503098 +0000 UTC m=+1143.367203109"
Nov 03 22:42:20.603278 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.603244    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:20.604788 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.604762    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:42:20.604880 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.604815    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:42:20.906672 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.906587    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:42:20.906847 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.906689    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:42:20.919617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.918777    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" event={"ID":"037e749e-a34d-4610-a3da-9eaef32a46ab","Type":"ContainerStarted","Data":"bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651"}
Nov 03 22:42:20.919617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:20.918847    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" event={"ID":"037e749e-a34d-4610-a3da-9eaef32a46ab","Type":"ContainerStarted","Data":"b8fa7a8dfb655857e962df9bbbb4e24d2f9e1d0cd24926c69869d5f3c57784c9"}
Nov 03 22:42:21.091411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.090711    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podStartSLOduration=2.090689814 podStartE2EDuration="2.090689814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:42:21.048861035 +0000 UTC m=+1144.313561047" watchObservedRunningTime="2026-11-03 22:42:21.090689814 +0000 UTC m=+1144.355389826"
Nov 03 22:42:21.558414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.557538    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:21.559416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.559061    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:42:21.559416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.559112    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:42:21.602841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.602718    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:42:21.602841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.602780    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:42:21.913067 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.912530    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:42:21.913067 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:21.912592    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:42:22.561415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:22.559765    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:42:22.561415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:22.559826    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:42:22.603273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:22.603224    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:42:22.603483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:22.603287    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:42:22.729564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:22.729535    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:22.908472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:22.906834    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:42:22.908472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:22.906892    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:42:23.559480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:23.559112    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:42:23.559480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:23.559167    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:42:23.609181 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:23.609147    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:42:23.609536 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:23.609505    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:42:23.905474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:23.904794    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:42:23.905474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:23.904860    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:42:24.569416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:24.568453    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:42:24.569416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:24.568512    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:42:24.605415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:24.604671    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:42:24.605415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:24.604730    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:42:24.908993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:24.907757    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:42:24.908993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:24.907811    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:42:25.560030 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:25.559995    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:42:25.560316 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:25.560271    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:42:25.603431 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:25.603381    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:42:25.603985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:25.603956    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:42:25.904469 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:25.904348    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:42:25.904469 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:25.904434    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:42:26.557366 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:26.557325    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:42:26.557582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:26.557410    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:42:26.956858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:26.956819    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:26.958248 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:26.958228    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:26.958898 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:26.958878    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:42:27.172729 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:27.172639    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:27.173248 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:27.173226    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:27.173828 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:27.173806    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:42:27.846611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:27.846577    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:27.847886 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:27.847427    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:27.848561 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:27.848543    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:42:28.142124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:28.142049    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:34.379895 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:34.379863    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:39.475104 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:39.475066    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:40.084509 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:42:40.084475    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:42:44.432341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:44.432310    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:49.084193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:49.084154    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:49.323212 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:49.323174    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:54.154890 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:54.154857    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:54.518122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:54.518090    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:59.032677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:59.032645    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:42:59.234372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:42:59.234339    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:03.571725 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.571688    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"]
Nov 03 22:43:03.577200 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.577175    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.633408 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.633361    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"]
Nov 03 22:43:03.667650 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.667613    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-certs\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.667852 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.667656    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istiod-ca-cert\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.668076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.667938    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-data\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.668166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.668074    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-credential-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.668166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.668120    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-envoy\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.668166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.668152    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-token\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.668332 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.668222    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.668332 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.668261    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-podinfo\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.668509 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.668481    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96td9\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-kube-api-access-96td9\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769350    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-credential-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769411    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-envoy\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769676 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769437    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-token\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769676 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769506    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769676 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769543    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-podinfo\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769676 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769646    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96td9\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-kube-api-access-96td9\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769755    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-certs\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769781    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istiod-ca-cert\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.769963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.769909    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-data\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.770172 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.770138    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-credential-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.770280 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.770229    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-socket\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.770352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.770289    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-data\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.770464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.770441    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-certs\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.771149 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.771123    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istiod-ca-cert\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.772057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.772035    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-envoy\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.772653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.772632    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-podinfo\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.835618 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.835565    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96td9\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-kube-api-access-96td9\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.842630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.842606    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-token\") pod \"e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") " pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:03.890853 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:03.890813    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:04.063907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.063871    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:04.218723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.218683    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"]
Nov 03 22:43:04.223764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.223742    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.280322 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.280286    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"]
Nov 03 22:43:04.282577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.282547    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.282744 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.282599    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.282803 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.282765    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btvjl\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-kube-api-access-btvjl\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.282846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.282830    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.282913 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.282897    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.282983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.282968    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.283126 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.283110    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.283188 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.283150    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.283280 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.283242    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.320211 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:43:04.320180    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23c229a_024d_422e_9c71_2b2c4abdfbbd.slice/crio-654f22924b661d959f9fafd81cc69c8c5fdf7082204389894aa493376ff6db51 WatchSource:0}: Error finding container 654f22924b661d959f9fafd81cc69c8c5fdf7082204389894aa493376ff6db51: Status 404 returned error can't find the container with id 654f22924b661d959f9fafd81cc69c8c5fdf7082204389894aa493376ff6db51
Nov 03 22:43:04.323062 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.323027    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:04.323151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.323096    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:04.323151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.323136    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:04.331473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.331450    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"]
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.401626    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.401673    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.401778    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btvjl\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-kube-api-access-btvjl\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.401830    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.401870    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.401911    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.402040    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.402074    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.402124    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.402600    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.402831    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.403152    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.404041    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.404806    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.414588    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.414895    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.430213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.426061    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" event={"ID":"a23c229a-024d-422e-9c71-2b2c4abdfbbd","Type":"ContainerStarted","Data":"654f22924b661d959f9fafd81cc69c8c5fdf7082204389894aa493376ff6db51"}
Nov 03 22:43:04.472614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.472503    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btvjl\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-kube-api-access-btvjl\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.473973 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.473945    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.543411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.539982    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:04.928460 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.922358    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"]
Nov 03 22:43:04.948414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.947902    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:04.977648 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:04.977078    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"]
Nov 03 22:43:05.015197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015160    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk6mc\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-kube-api-access-gk6mc\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015251    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015356    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015384    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb20c55-3376-49c7-a038-2edaa64ec005-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015515    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015575    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015604    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015661    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.015783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.015754    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.037385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.035091    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"]
Nov 03 22:43:05.046646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.044552    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:05.046646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.044627    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:05.046646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.044666    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117218    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk6mc\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-kube-api-access-gk6mc\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117311    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117436    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117468    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb20c55-3376-49c7-a038-2edaa64ec005-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117549    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117613    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117650    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117701    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.117771    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.118361    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-credential-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.139445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.119017    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-data\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.140789 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.140762    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-certs\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.141582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.141562    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb20c55-3376-49c7-a038-2edaa64ec005-istiod-ca-cert\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.158833 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.143694    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-socket\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.185470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.181453    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-podinfo\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.201813 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.191452    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-envoy\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.222124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.222094    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk6mc\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-kube-api-access-gk6mc\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.226453 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.226409    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-token\") pod \"e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") " pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.299185 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.297715    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:05.523503 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.512934    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" event={"ID":"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b","Type":"ContainerStarted","Data":"a48e573e2ce4ade093523efbec9a73c69b8f84c070aac8710538537fe8c22449"}
Nov 03 22:43:05.523503 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.513049    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" event={"ID":"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b","Type":"ContainerStarted","Data":"fd1ce5d1f7fad4d88f8ed667af797d1b3c22522f65bd34b3276be77ffbf4d492"}
Nov 03 22:43:05.544416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.540133    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:05.571412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.571176    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" event={"ID":"a23c229a-024d-422e-9c71-2b2c4abdfbbd","Type":"ContainerStarted","Data":"89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0"}
Nov 03 22:43:05.690346 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.690275    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podStartSLOduration=1.6902551639999999 podStartE2EDuration="1.690255164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:05.641065581 +0000 UTC m=+1188.905765605" watchObservedRunningTime="2026-11-03 22:43:05.690255164 +0000 UTC m=+1188.954955172"
Nov 03 22:43:05.971434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.956452    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:05.971434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.967590    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:05.971434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:05.967648    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:06.012293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.012233    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podStartSLOduration=3.012211888 podStartE2EDuration="3.012211888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:05.787857202 +0000 UTC m=+1189.052557214" watchObservedRunningTime="2026-11-03 22:43:06.012211888 +0000 UTC m=+1189.276911901"
Nov 03 22:43:06.359698 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.294860    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/health\": context deadline exceeded"
Nov 03 22:43:06.398124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.396944    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Nov 03 22:43:06.496600 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.494534    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:43:06.498128 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.498097    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 03 22:43:06.521633 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.521511    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:06.523096 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.522090    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:06.555414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.542839    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:06.555414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.542901    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:06.644417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.616600    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"]
Nov 03 22:43:06.644417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.622475    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:06.644417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.624049    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 03 22:43:06.713685 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:43:06.687831    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fb20c55_3376_49c7_a038_2edaa64ec005.slice/crio-6e0b73e984dc3f67cda73e3da82e39c7496bd5929416bcaec6f863acce8ae019 WatchSource:0}: Error finding container 6e0b73e984dc3f67cda73e3da82e39c7496bd5929416bcaec6f863acce8ae019: Status 404 returned error can't find the container with id 6e0b73e984dc3f67cda73e3da82e39c7496bd5929416bcaec6f863acce8ae019
Nov 03 22:43:06.734966 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.734911    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:06.735226 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.735203    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:06.735438 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.735417    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:06.756473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.754152    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:06.784422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.783775    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:06.837416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.832512    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:43:06.901415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.897714    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:06.901707 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:06.901675    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:07.040193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.040153    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.057590 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.052161    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.244455 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.244189    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:43:07.309424 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.309084    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.412986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.405753    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Nov 03 22:43:07.412986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.406234    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:43:07.497417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.494858    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.497417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.495225    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:43:07.511564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.510065    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.511564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.510468    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:43:07.532781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.532736    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:43:07.533371 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.533337    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.533849 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.533830    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:43:07.534016 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.534003    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:43:07.661487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.632524    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.661487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.633370    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.661487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.633720    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:43:07.661487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.633784    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:43:07.661487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.653644    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:07.661487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.653694    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:07.801416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.800713    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 03 22:43:07.801591 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.801569    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 03 22:43:07.813498 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.801909    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:43:07.813498 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.801972    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:43:07.864415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.846925    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:07.864415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.847295    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:43:07.929806 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.918601    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:07.929806 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:07.918652    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:08.056416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.052494    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.056416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.052938    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:43:08.056997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.056441    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 03 22:43:08.060917 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.060515    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:43:08.105136 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.105100    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" event={"ID":"3fb20c55-3376-49c7-a038-2edaa64ec005","Type":"ContainerStarted","Data":"2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3"}
Nov 03 22:43:08.106485 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.106457    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" event={"ID":"3fb20c55-3376-49c7-a038-2edaa64ec005","Type":"ContainerStarted","Data":"6e0b73e984dc3f67cda73e3da82e39c7496bd5929416bcaec6f863acce8ae019"}
Nov 03 22:43:08.298470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.272988    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.298470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.273377    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:43:08.371417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.327073    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:08.371417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.327181    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.371417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.327597    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:43:08.424428 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.424372    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.520510 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.502459    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.545529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.544748    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:08.545529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.544803    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:08.545529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.544881    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:08.545529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.544900    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:08.602457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.547840    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:43:08.602457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.574553    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.602457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.591233    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.870910 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.858806    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.870910 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.859225    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:08.870910 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:08.859545    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.170475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.133866    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.170475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.138644    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:43:09.302722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.302675    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 03 22:43:09.310416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.304028    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:43:09.438689 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.438582    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": context deadline exceeded"
Nov 03 22:43:09.512698 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.512652    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.520413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.519446    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.521973 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.521927    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:09.522251 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.522222    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:09.524306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.524247    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.620480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.569067    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.620480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.618806    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:09.620480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.618874    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:09.838515 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.837913    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.908861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.839954    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.909685 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.840491    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:09.910336 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.864474    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 03 22:43:09.926425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.881447    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:09.926425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.925501    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:09.926425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.912555    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:09.926425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.925569    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:09.940373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.940321    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949747    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949796    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949828    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949861    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949893    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949927    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949959    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:43:09.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:09.949990    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:43:10.111522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.111402    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podStartSLOduration=6.111366991 podStartE2EDuration="6.111366991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:09.988018379 +0000 UTC m=+1193.252718390" watchObservedRunningTime="2026-11-03 22:43:10.111366991 +0000 UTC m=+1193.376067005"
Nov 03 22:43:10.158418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.129675    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:43:10.158418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.139580    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:43:10.347532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.344567    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:43:10.354019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.352167    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:10.354019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.352220    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:10.359457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.354935    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:43:10.404304 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:43:10.403614    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:43:10.426553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.425807    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:43:10.625608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.625424    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:10.626319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.626283    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:10.937182 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.937149    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:10.937462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:10.937435    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:11.033035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:11.032911    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:11.387475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:11.356554    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:11.387475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:11.356612    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:11.623170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:11.623023    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:11.623170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:11.623078    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:11.922516 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:11.919017    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:11.934574 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:11.934509    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:12.334443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:12.334185    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:12.335800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:12.335766    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:12.628506 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:12.566514    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:12.628506 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:12.566582    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:12.914023 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:12.913928    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:12.914321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:12.914264    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:13.319704 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:13.319412    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:13.319704 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:13.319462    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:13.600635 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:13.600541    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:13.600885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:13.600855    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:13.891745 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:13.891662    5162 patch_prober.go:29] interesting pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:13.891745 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:13.891728    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:14.298652 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:14.298616    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:14.298834 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:14.298680    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:14.540512 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:14.540474    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:14.540720 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:14.540535    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:14.897173 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:14.897144    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:14.898378 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:14.898349    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:14.899282 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:14.899263    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:43:15.298575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:15.298541    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:15.298743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:15.298605    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:15.545655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:15.545626    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:15.545967 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:15.545940    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:15.546798 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:15.546776    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:43:16.135447 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:16.135329    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:16.304871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:16.304839    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:16.305617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:16.305593    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:16.306467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:16.306447    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:43:21.033351 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:21.033312    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:25.729541 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:25.729507    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:25.941157 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:25.941119    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:26.158343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:26.158264    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:28.309772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.309739    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"]
Nov 03 22:43:28.314170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.314148    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.361065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.361038    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"]
Nov 03 22:43:28.365195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.365174    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.412874 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.412848    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"]
Nov 03 22:43:28.429316 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429285    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.429514 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429463    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.429587 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429548    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l7gr\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-kube-api-access-9l7gr\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.429680 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429649    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.429739 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429693    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.429799 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429761    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.429857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429803    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.429857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429842    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.429942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429862    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.429942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429880    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.429942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429896    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.429942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429934    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.430079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429950    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.430079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429974    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszbr\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-kube-api-access-tszbr\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.430079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.429990    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.430079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.430014    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8826e5c8-7b53-4394-8f91-df31cab2db1d-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.430079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.430043    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.430292 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.430122    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.485516 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.485486    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"]
Nov 03 22:43:28.532361 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.532323    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.533055 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533006    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.533055 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533051    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.533277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533082    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.533277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533107    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.533277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533162    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.533277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533185    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.533277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533222    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszbr\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-kube-api-access-tszbr\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.533277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533249    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533286    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8826e5c8-7b53-4394-8f91-df31cab2db1d-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533327    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533466    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533550    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533589    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533673    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7gr\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-kube-api-access-9l7gr\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533729    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533759    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.534140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.533819    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.534677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.534169    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.534677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.534463    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.535303 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.535280    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8826e5c8-7b53-4394-8f91-df31cab2db1d-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.535542 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.535524    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.536790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.536489    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.537357 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.537331    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.537547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.537525    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.537656 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.537637    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istiod-ca-cert\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.537749 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.537658    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-credential-socket\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.537914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.537890    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-certs\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.538203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.538183    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-data\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.541202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.540219    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.541202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.540703    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-envoy\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.542023 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.542002    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-podinfo\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.598221 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.598142    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.598372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.598333    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszbr\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-kube-api-access-tszbr\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.598372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.598353    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-token\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:28.599218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.599196    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l7gr\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-kube-api-access-9l7gr\") pod \"e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") " pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.631678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.631653    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:28.678048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:28.678016    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:29.054677 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:43:29.054648    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7bd07ac_ea7b_4be4_8c87_35d96d7bce72.slice/crio-6c14fd4ad946ef45c0c734f497106224f92a1d33718167df186235366edc65f6 WatchSource:0}: Error finding container 6c14fd4ad946ef45c0c734f497106224f92a1d33718167df186235366edc65f6: Status 404 returned error can't find the container with id 6c14fd4ad946ef45c0c734f497106224f92a1d33718167df186235366edc65f6
Nov 03 22:43:29.057334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.057296    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:29.057476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.057367    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:29.057476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.057429    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:29.058849 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.058827    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"]
Nov 03 22:43:29.105730 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:43:29.105705    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8826e5c8_7b53_4394_8f91_df31cab2db1d.slice/crio-ea0bb0db4b738594a56dc2df2ef9aed8a9ccf9674285ebcd03eb886c08946617 WatchSource:0}: Error finding container ea0bb0db4b738594a56dc2df2ef9aed8a9ccf9674285ebcd03eb886c08946617: Status 404 returned error can't find the container with id ea0bb0db4b738594a56dc2df2ef9aed8a9ccf9674285ebcd03eb886c08946617
Nov 03 22:43:29.108466 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.108379    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:29.108559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.108541    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:29.108632 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.108593    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:29.110220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.110200    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"]
Nov 03 22:43:29.193091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.193058    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" event={"ID":"8826e5c8-7b53-4394-8f91-df31cab2db1d","Type":"ContainerStarted","Data":"ea0bb0db4b738594a56dc2df2ef9aed8a9ccf9674285ebcd03eb886c08946617"}
Nov 03 22:43:29.195013 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.194985    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" event={"ID":"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72","Type":"ContainerStarted","Data":"a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931"}
Nov 03 22:43:29.195161 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.195035    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" event={"ID":"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72","Type":"ContainerStarted","Data":"6c14fd4ad946ef45c0c734f497106224f92a1d33718167df186235366edc65f6"}
Nov 03 22:43:29.328112 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.325245    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podStartSLOduration=1.325228523 podStartE2EDuration="1.325228523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:29.316320579 +0000 UTC m=+1212.581020594" watchObservedRunningTime="2026-11-03 22:43:29.325228523 +0000 UTC m=+1212.589928537"
Nov 03 22:43:29.451377 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.451327    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podStartSLOduration=1.451313774 podStartE2EDuration="1.451313774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:29.450856982 +0000 UTC m=+1212.715557002" watchObservedRunningTime="2026-11-03 22:43:29.451313774 +0000 UTC m=+1212.716013785"
Nov 03 22:43:29.635048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.634961    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:29.678624 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:29.678584    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:30.205537 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:30.205502    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" event={"ID":"8826e5c8-7b53-4394-8f91-df31cab2db1d","Type":"ContainerStarted","Data":"617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f"}
Nov 03 22:43:30.633848 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:30.633760    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:43:30.633848 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:30.633816    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:43:30.683414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:30.682857    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:43:30.683414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:30.682917    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:43:30.964416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:30.963689    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:31.632722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:31.632689    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:43:31.632928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:31.632762    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:43:31.689414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:31.688784    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:43:31.689414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:31.688842    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:43:31.939805 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:31.939772    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:32.190473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:32.190097    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:32.632564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:32.632484    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:43:32.632564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:32.632548    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:43:32.689465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:32.686292    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:43:32.689465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:32.686351    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:43:32.763415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:32.760771    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:33.632170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:33.632137    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:43:33.632341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:33.632185    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:43:33.684793 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:33.684757    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:43:33.685072 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:33.685043    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:43:34.478550 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.478478    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"]
Nov 03 22:43:34.484928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.484902    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.501523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.501500    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f45de95-189b-4eeb-9dce-1d58d1f01359-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.501738 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.501713    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.501828 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.501812    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.501874 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.501847    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lghkd\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-kube-api-access-lghkd\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.501923 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.501907    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.501967 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.501955    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.502002 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.501982    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.502039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.502003    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.502039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.502030    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.609450 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.609383    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.609732 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.609703    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lghkd\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-kube-api-access-lghkd\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.609894 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.609880    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.610078 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.610061    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.610230 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.610216    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.610384 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.610369    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.611060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.610535    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.611060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.610732    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f45de95-189b-4eeb-9dce-1d58d1f01359-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.611060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.610911    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.611521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.611497    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.611756 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.611737    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.613277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.613251    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f45de95-189b-4eeb-9dce-1d58d1f01359-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.613526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.613505    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.613742 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.613724    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.619110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.615583    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.619484 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.619463    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"]
Nov 03 22:43:34.619937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.619889    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.632364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.632023    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:43:34.632364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.632073    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:43:34.698963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.698921    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:43:34.699156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.698985    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:43:34.702665 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.702635    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.704338 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.702878    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lghkd\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-kube-api-access-lghkd\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:34.808093 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:34.808011    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:35.302037 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.302000    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"]
Nov 03 22:43:35.334792 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.334756    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:35.334914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.334831    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:35.334914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.334880    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:43:35.632760 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.632669    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:43:35.633192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.632755    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:43:35.679241 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.679201    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:43:35.679428 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:35.679269    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:43:36.268605 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.268565    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" event={"ID":"9f45de95-189b-4eeb-9dce-1d58d1f01359","Type":"ContainerStarted","Data":"9241ef14a8e7b9670929b94799efe018d7cf227a0fc0c5f4d40949ac82d47e9b"}
Nov 03 22:43:36.268903 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.268878    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" event={"ID":"9f45de95-189b-4eeb-9dce-1d58d1f01359","Type":"ContainerStarted","Data":"8457e0803a4fe4490c1fb42c99042f6161671f5d3537c4a87621ff713e67c1ee"}
Nov 03 22:43:36.419985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.419933    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:36.577464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.552051    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podStartSLOduration=2.5520319320000002 podStartE2EDuration="2.552031932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:36.419756315 +0000 UTC m=+1219.684456329" watchObservedRunningTime="2026-11-03 22:43:36.552031932 +0000 UTC m=+1219.816731943"
Nov 03 22:43:36.599085 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.599015    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"]
Nov 03 22:43:36.612889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.612847    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy" containerID="cri-o://ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6" gracePeriod=30
Nov 03 22:43:36.775734 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.775701    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:36.777244 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.777221    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:36.779818 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.779798    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:43:36.794907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.794858    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:36.795313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.795295    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:36.797355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.797337    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:43:36.808373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.808355    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:36.809969 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.809951    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:43:36.810065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:36.809994    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:43:37.808822 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:37.808781    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:43:37.809202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:37.808844    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:43:38.809178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:38.809144    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:43:38.809613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:38.809202    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:43:39.090300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:39.090225    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:39.428723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:39.428683    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:39.636339 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:39.636302    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:39.825368 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:39.825280    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:43:39.827324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:39.827291    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:43:39.863933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:39.863901    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:40.083476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:40.083364    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:40.092497 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:43:40.092460    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:43:40.592596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:40.592559    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:40.811542 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:40.811509    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:43:40.811840 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:40.811808    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:43:41.810264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:41.810226    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:43:41.810839 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:41.810305    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:43:41.957433 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:41.957407    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:43:42.037038 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037009    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-kube-api-access-z8gcv\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-kube-api-access-z8gcv\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037207 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037099    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-certs\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037207 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037191    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istiod-ca-cert\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037229    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-credential-socket\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037269    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-data\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037428 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037330    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-token\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-token\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037478    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-certs" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:43:42.037547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037512    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-podinfo\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037551    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-credential-socket" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:43:42.037604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037569    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-socket\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037572    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-data" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:43:42.037761 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037621    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-envoy\") pod \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\" (UID: \"f47a8d9d-0538-459c-9bf4-0d46e30d8794\") "
Nov 03 22:43:42.037761 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037710    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istiod-ca-cert" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:43:42.037869 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.037779    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-socket" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:43:42.038372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.038346    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.038526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.038375    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.038526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.038407    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.038526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.038421    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.038526 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.038435    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.039736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.039705    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-token" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:43:42.039846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.039790    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-podinfo" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:43:42.039924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.039840    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-kube-api-access-z8gcv" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "kube-api-access-z8gcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:43:42.039924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.039904    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-envoy" pod "f47a8d9d-0538-459c-9bf4-0d46e30d8794" (UID: "f47a8d9d-0538-459c-9bf4-0d46e30d8794"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:43:42.140247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.140210    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.140247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.140245    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.140534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.140260    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8gcv\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-kube-api-access-z8gcv\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.140534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.140275    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f47a8d9d-0538-459c-9bf4-0d46e30d8794-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:43:42.281276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.281192    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:42.319587 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.319548    5162 generic.go:356] "Generic (PLEG): container finished" podID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerID="ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6" exitCode=0
Nov 03 22:43:42.319780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.319732    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" event={"ID":"f47a8d9d-0538-459c-9bf4-0d46e30d8794","Type":"ContainerDied","Data":"ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6"}
Nov 03 22:43:42.319846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.319815    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx" event={"ID":"f47a8d9d-0538-459c-9bf4-0d46e30d8794","Type":"ContainerDied","Data":"e783c126308e932dd982c9f1414299f730fde5cad82abea9ce68394913deb34c"}
Nov 03 22:43:42.319905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.319844    5162 scope.go:122] "RemoveContainer" containerID="ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6"
Nov 03 22:43:42.320024 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.320007    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"
Nov 03 22:43:42.350239 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.350215    5162 scope.go:122] "RemoveContainer" containerID="ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6"
Nov 03 22:43:42.350706 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:43:42.350605    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6\": container with ID starting with ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6 not found: ID does not exist" containerID="ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6"
Nov 03 22:43:42.350706 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.350642    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6"} err="failed to get container status \"ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6\": rpc error: code = NotFound desc = could not find container \"ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6\": container with ID starting with ac8bf757e36d402162a5f3a0398c1b4805d0959a6d0f296cfb5e2fd1b3de4fa6 not found: ID does not exist"
Nov 03 22:43:42.500583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.500551    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"]
Nov 03 22:43:42.504235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.504208    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx"]
Nov 03 22:43:42.531066 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.531033    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:42.858195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:42.858166    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" path="/var/lib/kubelet/pods/f47a8d9d-0538-459c-9bf4-0d46e30d8794/volumes"
Nov 03 22:43:43.045440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:43.045407    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:43.045831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:43.045810    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:43.047236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:43.047214    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 22:43:44.065822 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:44.065780    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:44.428066 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:44.428036    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:44.686141 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:44.686048    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:46.483740 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:46.483703    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:46.737926 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:46.732160    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:48.047420 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.032993    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 03 22:43:48.047420 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.043820    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:43:48.150603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.150556    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:48.154124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.151572    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 03 22:43:48.154124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.151611    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 03 22:43:48.323471 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.323244    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:48.505535 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.505484    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:48.535532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.535475    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:48.551517 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.551471    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:48.595474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.587642    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Nov 03 22:43:48.626524 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:48.626476    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.058461 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.056981    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 03 22:43:49.058461 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.057804    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:43:49.059111 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.059084    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.060131 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.059383    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:43:49.105416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.102265    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.160536 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.158514    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.164531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.160966    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.164531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.161306    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.164531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.161598    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 03 22:43:49.164531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.161903    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:43:49.164531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.161969    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:43:49.164531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.162006    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:43:49.164531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.164265    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.165381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.165082    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.503751 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.413702    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.503751 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.414078    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:43:49.503751 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.475228    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": context deadline exceeded"
Nov 03 22:43:49.531417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.505905    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:43:49.542810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.542696    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.555534 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.555421    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.576419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.571356    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:43:49.604443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.598094    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Nov 03 22:43:49.605309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.605279    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:43:49.631441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.621901    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:43:49.631441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.621991    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:43:49.632334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.632300    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:49.650606 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.650578    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:43:49.758547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:49.757207    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:50.125470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.112278    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:43:50.150910 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.138411    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:50.151340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.142145    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:50.173576 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.173527    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:43:50.176311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.176280    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 03 22:43:50.177752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.177726    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:43:50.178944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.178919    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 03 22:43:50.180552 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.180530    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:43:50.220414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.199892    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Nov 03 22:43:50.220414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.201660    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:43:50.220414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.205025    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:50.234420 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.224673    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:43:50.439475 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.429694    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:43:50.517463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.515876    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:50.572418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.572119    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:50.581638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.581600    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:50.592565 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.592502    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:50.620552 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.620515    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:50.636520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:50.636478    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 03 22:43:51.220246 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.220182    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:51.238442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.218621    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 03 22:43:51.247544 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.246902    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:51.247544 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.247111    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:43:51.247544 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.247233    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Nov 03 22:43:51.247544 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.247457    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:43:51.247918 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.247743    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 03 22:43:51.248432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.248405    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:43:51.288339 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.288308    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:43:51.313754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.313713    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:51.353127 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.350573    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:43:51.455412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.433358    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:51.503363 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.503231    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:43:51.504840 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.504709    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 22:43:51.505682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.505637    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:43:51.531473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.529569    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 22:43:51.531473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.529623    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:43:51.531473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.529655    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:43:51.553528 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.553500    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp"
Nov 03 22:43:51.582491 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.582455    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:51.583873 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.583753    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:43:51.597406 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.593898    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 22:43:51.617503 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.616727    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:43:51.625358 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.624197    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 22:43:51.628341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:51.628322    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:43:55.098813 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:55.098777    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:56.759337 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:56.759303    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:56.966916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:56.966878    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:59.420093 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:59.420048    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:43:59.618494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:43:59.618440    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:00.222441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:00.222382    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:00.526885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:00.526800    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:00.803626 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:00.803525    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:01.302991 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.302955    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:01.503320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.503278    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:01.805820 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.805788    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"]
Nov 03 22:44:01.806666 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.806643    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy"
Nov 03 22:44:01.806666 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.806665    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy"
Nov 03 22:44:01.806869 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.806855    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f47a8d9d-0538-459c-9bf4-0d46e30d8794" containerName="istio-proxy"
Nov 03 22:44:01.810692 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.810673    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:01.868276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.868244    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"]
Nov 03 22:44:01.908314 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.908282    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2x8h\" (UniqueName: \"kubernetes.io/projected/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-kube-api-access-b2x8h\") pod \"payload-processing-e2e-ait-bb95c832-584887d746-5cw9h\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") " pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:01.908560 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.908534    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb95c832-584887d746-5cw9h\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") " pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:01.912355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.912321    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bb95c832\""
Nov 03 22:44:01.912490 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.912417    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bb95c832-dockercfg-fl9s6\""
Nov 03 22:44:01.986495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.986459    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"]
Nov 03 22:44:01.990840 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.990817    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:01.991920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.991902    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"]
Nov 03 22:44:01.996080 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:01.996063    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.010001 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.009977    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb95c832-584887d746-5cw9h\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") " pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:02.010134 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.010122    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2x8h\" (UniqueName: \"kubernetes.io/projected/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-kube-api-access-b2x8h\") pod \"payload-processing-e2e-ait-bb95c832-584887d746-5cw9h\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") " pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:02.010670 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.010654    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb95c832-584887d746-5cw9h\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") " pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:02.042071 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.042037    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"]
Nov 03 22:44:02.052266 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.052233    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c6905007-openshift-default-dockercfg-b5pgq\""
Nov 03 22:44:02.072959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.072891    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2x8h\" (UniqueName: \"kubernetes.io/projected/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-kube-api-access-b2x8h\") pod \"payload-processing-e2e-ait-bb95c832-584887d746-5cw9h\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") " pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:02.083811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.083783    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"]
Nov 03 22:44:02.111108 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111079    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t6s9\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-kube-api-access-5t6s9\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111271 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111133    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4zxk\" (UniqueName: \"kubernetes.io/projected/844839e5-c3c6-447e-af61-ccf4eb0edda5-kube-api-access-f4zxk\") pod \"payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:02.111323 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111277    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf015953-06ee-4239-b7e9-ed60c868e307-istio-podinfo\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111370 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111323    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-istio-token\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111463    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf015953-06ee-4239-b7e9-ed60c868e307-istiod-ca-cert\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111587 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111573    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-socket\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111667 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111654    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-credential-socket\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111750 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111734    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844839e5-c3c6-447e-af61-ccf4eb0edda5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:02.111799 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111782    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-certs\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111840 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111825    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-envoy\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.111881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.111848    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-data\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.124573 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.124547    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:02.140103 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.140075    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:02.217415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217255    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-socket\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.217415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217309    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-credential-socket\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.217415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217411    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844839e5-c3c6-447e-af61-ccf4eb0edda5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:02.217716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217442    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-certs\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.217716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217500    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-envoy\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.217716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217526    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-data\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.217716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217612    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t6s9\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-kube-api-access-5t6s9\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.217716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217676    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4zxk\" (UniqueName: \"kubernetes.io/projected/844839e5-c3c6-447e-af61-ccf4eb0edda5-kube-api-access-f4zxk\") pod \"payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217783    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf015953-06ee-4239-b7e9-ed60c868e307-istio-podinfo\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217842    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-istio-token\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.217882    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf015953-06ee-4239-b7e9-ed60c868e307-istiod-ca-cert\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.219195    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf015953-06ee-4239-b7e9-ed60c868e307-istiod-ca-cert\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.219524    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-data\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.221022    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-credential-socket\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.221320    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-socket\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.222598    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844839e5-c3c6-447e-af61-ccf4eb0edda5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:02.238206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.222851    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-certs\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.337505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.335490    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf015953-06ee-4239-b7e9-ed60c868e307-istio-podinfo\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.337505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.336617    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-envoy\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.337505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.337076    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t6s9\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-kube-api-access-5t6s9\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.343877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.338786    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4zxk\" (UniqueName: \"kubernetes.io/projected/844839e5-c3c6-447e-af61-ccf4eb0edda5-kube-api-access-f4zxk\") pod \"payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:02.350410 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.347862    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-istio-token\") pod \"e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") " pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.606266 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.606180    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:02.611667 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.611648    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:02.669529 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.669446    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:02.707715 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:44:02.707680    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3dc7acf_7733_47cc_9cbc_8a94b2a56b1e.slice/crio-4a8a08e4d0678d5c7bb6d32d89cb512466b39be0e3503ad79676e04f0f859e62 WatchSource:0}: Error finding container 4a8a08e4d0678d5c7bb6d32d89cb512466b39be0e3503ad79676e04f0f859e62: Status 404 returned error can't find the container with id 4a8a08e4d0678d5c7bb6d32d89cb512466b39be0e3503ad79676e04f0f859e62
Nov 03 22:44:02.711905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.711875    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"]
Nov 03 22:44:02.984554 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:44:02.984525    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844839e5_c3c6_447e_af61_ccf4eb0edda5.slice/crio-b52c17ea57b2e2fe6f9fe9698f5a37221ac0d1f47fd7df399c635bb4a1b35711 WatchSource:0}: Error finding container b52c17ea57b2e2fe6f9fe9698f5a37221ac0d1f47fd7df399c635bb4a1b35711: Status 404 returned error can't find the container with id b52c17ea57b2e2fe6f9fe9698f5a37221ac0d1f47fd7df399c635bb4a1b35711
Nov 03 22:44:02.989195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:02.989173    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"]
Nov 03 22:44:03.037830 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:44:03.037802    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf015953_06ee_4239_b7e9_ed60c868e307.slice/crio-0df219d8ff244cbf7a9e533b3377e8e8091677eb1f52c1c315af3b08f3a38f04 WatchSource:0}: Error finding container 0df219d8ff244cbf7a9e533b3377e8e8091677eb1f52c1c315af3b08f3a38f04: Status 404 returned error can't find the container with id 0df219d8ff244cbf7a9e533b3377e8e8091677eb1f52c1c315af3b08f3a38f04
Nov 03 22:44:03.040270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.040238    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:44:03.040344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.040304    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:44:03.040344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.040331    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:44:03.040893 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.040869    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"]
Nov 03 22:44:03.568876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.568811    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q" event={"ID":"844839e5-c3c6-447e-af61-ccf4eb0edda5","Type":"ContainerStarted","Data":"c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8"}
Nov 03 22:44:03.568876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.568873    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q" event={"ID":"844839e5-c3c6-447e-af61-ccf4eb0edda5","Type":"ContainerStarted","Data":"b52c17ea57b2e2fe6f9fe9698f5a37221ac0d1f47fd7df399c635bb4a1b35711"}
Nov 03 22:44:03.569228 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.569211    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:03.570714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.570688    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h" event={"ID":"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e","Type":"ContainerStarted","Data":"8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee"}
Nov 03 22:44:03.570843 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.570728    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h" event={"ID":"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e","Type":"ContainerStarted","Data":"4a8a08e4d0678d5c7bb6d32d89cb512466b39be0e3503ad79676e04f0f859e62"}
Nov 03 22:44:03.570843 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.570752    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:03.572106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.572086    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" event={"ID":"cf015953-06ee-4239-b7e9-ed60c868e307","Type":"ContainerStarted","Data":"5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a"}
Nov 03 22:44:03.572384 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.572126    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" event={"ID":"cf015953-06ee-4239-b7e9-ed60c868e307","Type":"ContainerStarted","Data":"0df219d8ff244cbf7a9e533b3377e8e8091677eb1f52c1c315af3b08f3a38f04"}
Nov 03 22:44:03.612421 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.612359    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:03.688976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.688922    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q" podStartSLOduration=2.6889082760000003 podStartE2EDuration="2.688908276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:03.687503923 +0000 UTC m=+1246.952203934" watchObservedRunningTime="2026-11-03 22:44:03.688908276 +0000 UTC m=+1246.953608309"
Nov 03 22:44:03.807927 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.807879    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h" podStartSLOduration=2.807864193 podStartE2EDuration="2.807864193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:03.806435015 +0000 UTC m=+1247.071135025" watchObservedRunningTime="2026-11-03 22:44:03.807864193 +0000 UTC m=+1247.072564208"
Nov 03 22:44:03.931099 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:03.931034    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podStartSLOduration=2.931017256 podStartE2EDuration="2.931017256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:03.930261591 +0000 UTC m=+1247.194961604" watchObservedRunningTime="2026-11-03 22:44:03.931017256 +0000 UTC m=+1247.195717269"
Nov 03 22:44:04.132667 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:04.130907    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:04.362215 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:04.362121    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:04.618187 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:04.618111    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:05.509598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.509568    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"]
Nov 03 22:44:05.514319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.514290    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.569102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.569074    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"]
Nov 03 22:44:05.590352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.590323    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:05.591163 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.591143    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:44:05.615749 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.615728    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bb95c832-serving-cert\""
Nov 03 22:44:05.674728 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.674698    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glsqm\" (UniqueName: \"kubernetes.io/projected/23923481-cfbe-4e62-8315-e0012658a189-kube-api-access-glsqm\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.674887 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.674820    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23923481-cfbe-4e62-8315-e0012658a189-openshift-service-ca\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.675130 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.675106    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-metrics-certs\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.675481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.675277    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-maas-api-tls\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.771612 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.771527    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:05.777091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.777069    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glsqm\" (UniqueName: \"kubernetes.io/projected/23923481-cfbe-4e62-8315-e0012658a189-kube-api-access-glsqm\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.777206 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.777170    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23923481-cfbe-4e62-8315-e0012658a189-openshift-service-ca\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.777383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.777348    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-metrics-certs\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.777517 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.777500    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-maas-api-tls\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.777831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.777811    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23923481-cfbe-4e62-8315-e0012658a189-openshift-service-ca\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.779758 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.779736    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-metrics-certs\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.779889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.779872    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-maas-api-tls\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.842107 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.842075    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glsqm\" (UniqueName: \"kubernetes.io/projected/23923481-cfbe-4e62-8315-e0012658a189-kube-api-access-glsqm\") pod \"maas-api-e2e-ait-bb95c832-6646967857-7ckgv\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:05.976743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:05.976705    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:06.130425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:06.130325    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:06.191727 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:06.189010    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:06.511043 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:44:06.511001    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23923481_cfbe_4e62_8315_e0012658a189.slice/crio-dced73ef82579affb8b54388a43b10a6be3e8663d7dc8cf10b14f978948b3199 WatchSource:0}: Error finding container dced73ef82579affb8b54388a43b10a6be3e8663d7dc8cf10b14f978948b3199: Status 404 returned error can't find the container with id dced73ef82579affb8b54388a43b10a6be3e8663d7dc8cf10b14f978948b3199
Nov 03 22:44:06.512766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:06.512742    5162 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:44:06.515199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:06.515174    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"]
Nov 03 22:44:06.565940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:06.565913    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:06.599103 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:06.599070    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv" event={"ID":"23923481-cfbe-4e62-8315-e0012658a189","Type":"ContainerStarted","Data":"dced73ef82579affb8b54388a43b10a6be3e8663d7dc8cf10b14f978948b3199"}
Nov 03 22:44:06.781018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:06.780930    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:07.607720 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:07.607676    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv" event={"ID":"23923481-cfbe-4e62-8315-e0012658a189","Type":"ContainerStarted","Data":"d77ee6add35304f2c608e8ff5268e660b20046791a377ac75c14ac783e4d3b5e"}
Nov 03 22:44:07.608164 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:07.607917    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:07.728677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:07.728633    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv" podStartSLOduration=1.829729519 podStartE2EDuration="2.728618435s" totalImagesPullingTime="898.888916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:07.727641153 +0000 UTC m=+1250.992341168" watchObservedRunningTime="2026-11-03 22:44:07.728618435 +0000 UTC m=+1250.993318446"
Nov 03 22:44:08.845704 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:08.845667    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:09.057383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:09.057339    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:09.736722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:09.736688    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:10.088151 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:44:10.088061    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:44:11.347361 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:11.347321    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:13.216365 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:13.216325    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:13.621304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:13.621240    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:44:14.584293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:14.584263    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:44:14.584830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:14.584345    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:44:15.916593 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:15.916559    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"]
Nov 03 22:44:15.920572 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:15.920557    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:15.976074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:15.976043    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"]
Nov 03 22:44:16.022560 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.022508    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-027a6fbe-serving-cert\""
Nov 03 22:44:16.036157 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.036131    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.036267 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.036194    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-metrics-certs\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.036371 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.036350    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7375ed63-6334-4fa1-a2ca-a3db7444ae36-openshift-service-ca\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.036462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.036445    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtqh8\" (UniqueName: \"kubernetes.io/projected/7375ed63-6334-4fa1-a2ca-a3db7444ae36-kube-api-access-jtqh8\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.138571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.138533    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7375ed63-6334-4fa1-a2ca-a3db7444ae36-openshift-service-ca\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.138792 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.138584    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtqh8\" (UniqueName: \"kubernetes.io/projected/7375ed63-6334-4fa1-a2ca-a3db7444ae36-kube-api-access-jtqh8\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.138792 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.138715    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.138792 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.138764    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-metrics-certs\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.138969 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:44:16.138889    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-serving-cert: secret "maas-api-e2e-migrate-027a6fbe-serving-cert" not found
Nov 03 22:44:16.139025 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:44:16.138986    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls podName:7375ed63-6334-4fa1-a2ca-a3db7444ae36 nodeName:}" failed. No retries permitted until 2026-11-03 22:44:16.638964226 +0000 UTC m=+1259.903664231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls") pod "maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj" (UID: "7375ed63-6334-4fa1-a2ca-a3db7444ae36") : secret "maas-api-e2e-migrate-027a6fbe-serving-cert" not found
Nov 03 22:44:16.139167 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.139144    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7375ed63-6334-4fa1-a2ca-a3db7444ae36-openshift-service-ca\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.141042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.141021    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-metrics-certs\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.169162 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.169120    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:16.205766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.205725    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtqh8\" (UniqueName: \"kubernetes.io/projected/7375ed63-6334-4fa1-a2ca-a3db7444ae36-kube-api-access-jtqh8\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.384213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.384179    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:16.590620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.590534    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:16.647810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.647781    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.650109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.650085    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls\") pod \"maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:16.838644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:16.838611    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:17.288728 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:44:17.288701    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7375ed63_6334_4fa1_a2ca_a3db7444ae36.slice/crio-e971a4b0b9b46779652f29f93b0f404169741f528d3c7df5a5b32090b576034e WatchSource:0}: Error finding container e971a4b0b9b46779652f29f93b0f404169741f528d3c7df5a5b32090b576034e: Status 404 returned error can't find the container with id e971a4b0b9b46779652f29f93b0f404169741f528d3c7df5a5b32090b576034e
Nov 03 22:44:17.293586 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:17.293561    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"]
Nov 03 22:44:17.690173 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:17.690124    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj" event={"ID":"7375ed63-6334-4fa1-a2ca-a3db7444ae36","Type":"ContainerStarted","Data":"e971a4b0b9b46779652f29f93b0f404169741f528d3c7df5a5b32090b576034e"}
Nov 03 22:44:18.698707 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:18.698666    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj" event={"ID":"7375ed63-6334-4fa1-a2ca-a3db7444ae36","Type":"ContainerStarted","Data":"9ef2295c00dc632224b4c5253a32deb0a907b07719ebc6db59aeaf55114db416"}
Nov 03 22:44:18.699113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:18.698756    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:18.819616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:18.819559    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj" podStartSLOduration=2.880026266 podStartE2EDuration="3.819541874s" totalImagesPullingTime="939.515608ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:18.818865457 +0000 UTC m=+1262.083565467" watchObservedRunningTime="2026-11-03 22:44:18.819541874 +0000 UTC m=+1262.084241904"
Nov 03 22:44:18.967272 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:18.967194    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:19.179054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.179021    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:19.391473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.391356    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"]
Nov 03 22:44:19.396053 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.396027    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.417976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.417944    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418207    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40955271-29ca-4a4f-9c83-57ac8ab840de-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418271    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418347    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418514    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418562    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418619    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418659    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.418959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.418734    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k55dn\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-kube-api-access-k55dn\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.520003 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.519947    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.520678 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.520657    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.521237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.521219    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.521735 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.521689    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.521924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.521909    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k55dn\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-kube-api-access-k55dn\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.522416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.521621    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.522546 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.520588    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.522736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.522718    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.523170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.523152    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40955271-29ca-4a4f-9c83-57ac8ab840de-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.523367 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.523347    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.523833 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.523717    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.523922 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.523896    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40955271-29ca-4a4f-9c83-57ac8ab840de-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.524061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.524036    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.524402 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.524149    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.524525 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.524302    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.553411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.538419    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.565224 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.565141    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"]
Nov 03 22:44:19.586094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.585626    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.599691 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.599662    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k55dn\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-kube-api-access-k55dn\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.721025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.720989    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:19.890944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:19.890913    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:20.122702 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.122624    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:20.225868 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:44:20.225834    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40955271_29ca_4a4f_9c83_57ac8ab840de.slice/crio-73327e3f93352d0562777a28944becb6e51d9a3d6401bfe01001158e498407f4 WatchSource:0}: Error finding container 73327e3f93352d0562777a28944becb6e51d9a3d6401bfe01001158e498407f4: Status 404 returned error can't find the container with id 73327e3f93352d0562777a28944becb6e51d9a3d6401bfe01001158e498407f4
Nov 03 22:44:20.228631 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.228594    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:44:20.228710 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.228664    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:44:20.228710 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.228694    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:44:20.231831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.231809    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"]
Nov 03 22:44:20.716735 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.716697    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:20.719465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.719438    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" event={"ID":"40955271-29ca-4a4f-9c83-57ac8ab840de","Type":"ContainerStarted","Data":"ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17"}
Nov 03 22:44:20.719603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.719489    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" event={"ID":"40955271-29ca-4a4f-9c83-57ac8ab840de","Type":"ContainerStarted","Data":"73327e3f93352d0562777a28944becb6e51d9a3d6401bfe01001158e498407f4"}
Nov 03 22:44:20.721715 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.721690    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:20.845491 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.845435    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podStartSLOduration=1.845419192 podStartE2EDuration="1.845419192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:20.843029159 +0000 UTC m=+1264.107729170" watchObservedRunningTime="2026-11-03 22:44:20.845419192 +0000 UTC m=+1264.110119205"
Nov 03 22:44:20.925454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:20.925413    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:21.137348 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:21.137262    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:21.663974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:21.661528    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:21.724872 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:21.724842    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:44:21.725290 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:21.724895    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:44:22.722297 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:22.722261    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:44:22.722510 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:22.722329    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:44:23.466887 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:23.466849    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:23.686775 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:23.686737    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:23.722263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:23.722188    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:44:23.722263 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:23.722240    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:44:23.901716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:23.901682    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:24.166871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:24.166767    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:24.383637 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:24.383597    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:24.719900 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:24.719874    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:44:24.722743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:24.722717    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:44:24.722904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:24.722781    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:44:24.794255 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:24.794218    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:25.007465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:25.006889    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:25.722139 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:25.722105    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:44:25.722674 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:25.722641    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:44:26.382889 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:26.382855    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:26.722238 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:26.722201    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:44:26.722862 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:26.722830    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:44:27.809415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:27.809321    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:44:27.809926 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:27.809413    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:44:28.158468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.158258    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 03 22:44:28.182415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.178468    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:28.231413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.230495    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:28.256416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.249946    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:28.262523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.259457    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:28.308416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.308254    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:44:28.525655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.525607    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:28.617663 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.612509    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:28.636720 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:28.636035    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.177474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.113653    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:44:29.177474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.113708    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.179953 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.178142    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Nov 03 22:44:29.179953 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.178899    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.200441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.200383    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.240668 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.228047    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:44:29.240668 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.228118    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:44:29.278450 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.272168    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.278450 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.272648    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.302416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.283649    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:44:29.302416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.283725    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:44:29.306418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.304119    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:44:29.324592 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.308971    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.324592 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.313504    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:44:29.324592 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.313569    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:44:29.527456 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.527351    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.548632 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.548595    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:44:29.573866 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.573837    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:29.597342 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.597304    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:29.882463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.858601    5162 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": context deadline exceeded"
Nov 03 22:44:29.896822 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.896784    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:44:29.914001 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.913964    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp" podUID="9a63990d-9392-4c4a-b804-3dfa65e4a824" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 03 22:44:29.921811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.921789    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 22:44:29.947446 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.942357    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 22:44:29.959730 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.959678    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:29.972432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.966494    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:44:29.972432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.967900    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 03 22:44:30.005871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.972730    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:30.006170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.973156    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 22:44:30.006385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.006370    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 22:44:30.006551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.006541    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 22:44:30.006708 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.006686    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:44:30.006832 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:29.985473    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:30.008443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.007545    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:44:30.036171 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.036140    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 22:44:30.074427 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.069089    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 22:44:30.084187 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.084162    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 22:44:30.129696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.124549    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 22:44:30.419548 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.419512    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:30.704483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:30.694755    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:31.064671 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:31.064563    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:32.845945 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:32.845908    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:37.181433 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:37.181342    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:37.392688 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:37.392608    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:37.859073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:37.859033    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:40.088776 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:44:40.088740    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:44:42.732174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:42.732138    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:47.166175 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:47.166127    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:47.682709 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:47.682662    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:50.063084 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:50.063045    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:52.786166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:52.786122    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:53.141378 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:53.141280    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:57.344617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.344588    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"]
Nov 03 22:44:57.349217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.349199    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.404809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.404774    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"]
Nov 03 22:44:57.423891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.423859    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pswzx\" (UniqueName: \"kubernetes.io/projected/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-kube-api-access-pswzx\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.424007 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.423901    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-metrics-certs\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.424168 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.424146    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.424268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.424254    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-openshift-service-ca\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.451920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.451900    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c082ab23-serving-cert\""
Nov 03 22:44:57.525752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.525723    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.525931 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.525808    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-openshift-service-ca\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.525931 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.525854    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pswzx\" (UniqueName: \"kubernetes.io/projected/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-kube-api-access-pswzx\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.525931 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.525879    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-metrics-certs\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.526097 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:44:57.525879    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-serving-cert: secret "maas-api-e2e-ait-c082ab23-serving-cert" not found
Nov 03 22:44:57.526097 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:44:57.526024    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls podName:f0a66bd9-1980-42b2-8c2f-3fe2188a5f00 nodeName:}" failed. No retries permitted until 2026-11-03 22:44:58.026006348 +0000 UTC m=+1301.290706351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls") pod "maas-api-e2e-ait-c082ab23-797fc6564-h5flc" (UID: "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00") : secret "maas-api-e2e-ait-c082ab23-serving-cert" not found
Nov 03 22:44:57.526507 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.526484    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-openshift-service-ca\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.528241 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.528221    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-metrics-certs\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.589343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.589305    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:57.601205 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.601104    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pswzx\" (UniqueName: \"kubernetes.io/projected/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-kube-api-access-pswzx\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:57.808489 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:57.808444    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:58.010585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:58.010553    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:44:58.034276 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:58.034247    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:58.036765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:58.036740    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls\") pod \"maas-api-e2e-ait-c082ab23-797fc6564-h5flc\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:58.264434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:58.264311    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:44:58.639475 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:44:58.639440    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a66bd9_1980_42b2_8c2f_3fe2188a5f00.slice/crio-84da3e19d1b905d412842666057dce0a0d9393cbe34903b06dfb8dd5cfe785d8 WatchSource:0}: Error finding container 84da3e19d1b905d412842666057dce0a0d9393cbe34903b06dfb8dd5cfe785d8: Status 404 returned error can't find the container with id 84da3e19d1b905d412842666057dce0a0d9393cbe34903b06dfb8dd5cfe785d8
Nov 03 22:44:58.643744 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:58.643720    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"]
Nov 03 22:44:59.553617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:59.553576    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc" event={"ID":"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00","Type":"ContainerStarted","Data":"84da3e19d1b905d412842666057dce0a0d9393cbe34903b06dfb8dd5cfe785d8"}
Nov 03 22:44:59.811961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:44:59.811880    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:00.565318 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:00.565282    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc" event={"ID":"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00","Type":"ContainerStarted","Data":"ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04"}
Nov 03 22:45:00.566489 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:00.566458    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:45:00.689810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:00.689759    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc" podStartSLOduration=2.723461069 podStartE2EDuration="3.689745835s" totalImagesPullingTime="966.284766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:00.68730624 +0000 UTC m=+1303.952006251" watchObservedRunningTime="2026-11-03 22:45:00.689745835 +0000 UTC m=+1303.954445845"
Nov 03 22:45:03.451877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:03.451836    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:03.706135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:03.705686    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:04.108179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:04.108085    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:07.626248 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:07.626218    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:45:09.147584 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:09.146957    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:10.085568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:10.085531    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:10.085823 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:45:10.085691    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:45:14.337250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:14.337173    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:14.633821 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:14.633733    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:15.160665 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.160627    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"]
Nov 03 22:45:15.165081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.165061    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.228477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.228440    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"]
Nov 03 22:45:15.269777 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.269746    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-65cac0-openshift-default-dockercfg-4ws9v\""
Nov 03 22:45:15.278681 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.278654    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-certs\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.278845 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.278688    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-envoy\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.278845 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.278793    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-socket\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.278968 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.278856    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-data\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.278968 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.278940    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xk57\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-kube-api-access-7xk57\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.279085 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.279032    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9e36e5-5009-4773-9b24-42c4d14fa329-istiod-ca-cert\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.279085 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.279061    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-token\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.279194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.279141    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-credential-socket\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.279194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.279173    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-podinfo\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.301178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.301135    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:15.381535 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.381496    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-certs\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.382002 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.381984    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-envoy\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.395419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.382106    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-socket\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.395842 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.392443    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-socket\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.395949 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.393382    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-certs\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.396057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.396032    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-data\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.396199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.396180    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xk57\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-kube-api-access-7xk57\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.396413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.396382    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9e36e5-5009-4773-9b24-42c4d14fa329-istiod-ca-cert\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.396566 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.396550    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-token\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.396736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.396721    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-credential-socket\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.396866 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.396852    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-podinfo\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.402456 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.397839    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9e36e5-5009-4773-9b24-42c4d14fa329-istiod-ca-cert\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.403043 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.403005    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-data\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.403403 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.403362    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-credential-socket\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.416934 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.412383    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-envoy\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.432412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.428691    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-podinfo\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.461035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.460993    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-token\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.464277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.464248    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xk57\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-kube-api-access-7xk57\") pod \"e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") " pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.479370 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.479341    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:15.511602 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.507197    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:15.931911 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:45:15.931868    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9e36e5_5009_4773_9b24_42c4d14fa329.slice/crio-978e27aa82e6093a800627c37030e0ca04c8ecdc1d18bf81b6aa5a9042656416 WatchSource:0}: Error finding container 978e27aa82e6093a800627c37030e0ca04c8ecdc1d18bf81b6aa5a9042656416: Status 404 returned error can't find the container with id 978e27aa82e6093a800627c37030e0ca04c8ecdc1d18bf81b6aa5a9042656416
Nov 03 22:45:15.934725 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.934688    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:45:15.934831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.934772    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:45:15.934831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.934811    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:45:15.935521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:15.935500    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"]
Nov 03 22:45:16.696870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.696840    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"]
Nov 03 22:45:16.701071 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.701056    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.727165 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.727141    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mzfk\" (UniqueName: \"kubernetes.io/projected/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-kube-api-access-8mzfk\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.727311 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.727292    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.727404 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.727376    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.740902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.740877    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" event={"ID":"2c9e36e5-5009-4773-9b24-42c4d14fa329","Type":"ContainerStarted","Data":"ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8"}
Nov 03 22:45:16.741000 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.740928    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" event={"ID":"2c9e36e5-5009-4773-9b24-42c4d14fa329","Type":"ContainerStarted","Data":"978e27aa82e6093a800627c37030e0ca04c8ecdc1d18bf81b6aa5a9042656416"}
Nov 03 22:45:16.756575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.756543    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"]
Nov 03 22:45:16.801458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.801430    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 22:45:16.801560 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.801491    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjll9\""
Nov 03 22:45:16.828991 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.828958    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mzfk\" (UniqueName: \"kubernetes.io/projected/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-kube-api-access-8mzfk\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.829382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.829152    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.829888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.829864    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.830619 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.830602    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.882113 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.882093    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 22:45:16.891540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.891510    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.891722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.891703    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mzfk\" (UniqueName: \"kubernetes.io/projected/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-kube-api-access-8mzfk\") pod \"maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:16.949335 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:16.949243    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podStartSLOduration=1.949231333 podStartE2EDuration="1.949231333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:16.94625974 +0000 UTC m=+1320.210959751" watchObservedRunningTime="2026-11-03 22:45:16.949231333 +0000 UTC m=+1320.213931372"
Nov 03 22:45:17.074866 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.070782    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjll9\""
Nov 03 22:45:17.077785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.077756    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:17.479955 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.479930    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:17.480755 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:45:17.480726    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1e8790_7bb5_4b0d_81ad_5018ffcad95f.slice/crio-ea49c0cfb62f9ca932fa302629a4a430b25d034815fdbf816458435bb5d6ad6e WatchSource:0}: Error finding container ea49c0cfb62f9ca932fa302629a4a430b25d034815fdbf816458435bb5d6ad6e: Status 404 returned error can't find the container with id ea49c0cfb62f9ca932fa302629a4a430b25d034815fdbf816458435bb5d6ad6e
Nov 03 22:45:17.486862 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.486834    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"]
Nov 03 22:45:17.487645 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.487604    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:17.754554 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.754454    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m" event={"ID":"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f","Type":"ContainerStarted","Data":"ea49c0cfb62f9ca932fa302629a4a430b25d034815fdbf816458435bb5d6ad6e"}
Nov 03 22:45:17.754994 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.754707    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:17.755974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:17.755946    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:45:19.789247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:19.789216    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:20.054289 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:20.054195    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:22.791668 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:22.791637    5162 generic.go:356] "Generic (PLEG): container finished" podID="8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" containerID="f2cebec6358f60e37ca7fb1ce4ee06dd9659dfa4dbf03b8bb2dba0bb41f71c42" exitCode=0
Nov 03 22:45:22.792059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:22.791723    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m" event={"ID":"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f","Type":"ContainerDied","Data":"f2cebec6358f60e37ca7fb1ce4ee06dd9659dfa4dbf03b8bb2dba0bb41f71c42"}
Nov 03 22:45:23.084348 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:23.084275    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:23.363722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:23.363628    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:24.103954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.103931    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:24.157356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.157328    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-xks-ca\") pod \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") "
Nov 03 22:45:24.157598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.157381    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-kube-api-access-8mzfk\" (UniqueName: \"kubernetes.io/projected/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-kube-api-access-8mzfk\") pod \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") "
Nov 03 22:45:24.157598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.157432    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-openshift-ca\") pod \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\" (UID: \"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f\") "
Nov 03 22:45:24.157863 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.157841    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-openshift-ca" pod "8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" (UID: "8b1e8790-7bb5-4b0d-81ad-5018ffcad95f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:45:24.158211 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.158192    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-openshift-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:45:24.159594 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.159570    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-xks-ca" pod "8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" (UID: "8b1e8790-7bb5-4b0d-81ad-5018ffcad95f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:45:24.159740 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.159721    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-kube-api-access-8mzfk" pod "8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" (UID: "8b1e8790-7bb5-4b0d-81ad-5018ffcad95f"). InnerVolumeSpecName "kube-api-access-8mzfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:45:24.260054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.260023    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-maas-api-xks-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:45:24.260054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.260052    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mzfk\" (UniqueName: \"kubernetes.io/projected/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f-kube-api-access-8mzfk\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:45:24.811406 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.811362    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"
Nov 03 22:45:24.811604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.811412    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m" event={"ID":"8b1e8790-7bb5-4b0d-81ad-5018ffcad95f","Type":"ContainerDied","Data":"ea49c0cfb62f9ca932fa302629a4a430b25d034815fdbf816458435bb5d6ad6e"}
Nov 03 22:45:24.811604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.811463    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea49c0cfb62f9ca932fa302629a4a430b25d034815fdbf816458435bb5d6ad6e"
Nov 03 22:45:24.939148 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:24.939109    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:29.498947 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:29.498913    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"]
Nov 03 22:45:29.583531 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:29.583497    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"]
Nov 03 22:45:29.975415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:29.973465    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:30.208736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:30.208665    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:30.597586 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:30.597545    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:34.499871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.499825    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h" podUID="b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" containerName="payload-processing" containerID="cri-o://8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee" gracePeriod=25
Nov 03 22:45:34.582406 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:45:34.582368    5162 logging.go:55] [core] [Channel #294 SubChannel #296]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.80:50052", ServerName: "10.134.0.80:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.80:50052: connect: connection refused"
Nov 03 22:45:34.584228 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.584196    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q" podUID="844839e5-c3c6-447e-af61-ccf4eb0edda5" containerName="payload-pre-processing" containerID="cri-o://c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8" gracePeriod=25
Nov 03 22:45:34.883554 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.883524    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:45:34.914905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.914878    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:45:34.924091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.924060    5162 generic.go:356] "Generic (PLEG): container finished" podID="b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" containerID="8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee" exitCode=0
Nov 03 22:45:34.924218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.924096    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"
Nov 03 22:45:34.924218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.924128    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h" event={"ID":"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e","Type":"ContainerDied","Data":"8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee"}
Nov 03 22:45:34.924218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.924184    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h" event={"ID":"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e","Type":"ContainerDied","Data":"4a8a08e4d0678d5c7bb6d32d89cb512466b39be0e3503ad79676e04f0f859e62"}
Nov 03 22:45:34.924218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.924209    5162 scope.go:122] "RemoveContainer" containerID="8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee"
Nov 03 22:45:34.925963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.925942    5162 generic.go:356] "Generic (PLEG): container finished" podID="844839e5-c3c6-447e-af61-ccf4eb0edda5" containerID="c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8" exitCode=0
Nov 03 22:45:34.926064 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.925975    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q" event={"ID":"844839e5-c3c6-447e-af61-ccf4eb0edda5","Type":"ContainerDied","Data":"c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8"}
Nov 03 22:45:34.926064 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.926004    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"
Nov 03 22:45:34.926179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.926017    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q" event={"ID":"844839e5-c3c6-447e-af61-ccf4eb0edda5","Type":"ContainerDied","Data":"b52c17ea57b2e2fe6f9fe9698f5a37221ac0d1f47fd7df399c635bb4a1b35711"}
Nov 03 22:45:34.935585 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.935565    5162 scope.go:122] "RemoveContainer" containerID="8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee"
Nov 03 22:45:34.935866 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:45:34.935843    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee\": container with ID starting with 8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee not found: ID does not exist" containerID="8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee"
Nov 03 22:45:34.935946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.935871    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee"} err="failed to get container status \"8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee\": rpc error: code = NotFound desc = could not find container \"8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee\": container with ID starting with 8bc0facd900f6106b7be9249e524a336afee574b0d7b0c11c15af3c6fb76a4ee not found: ID does not exist"
Nov 03 22:45:34.935946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.935894    5162 scope.go:122] "RemoveContainer" containerID="c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8"
Nov 03 22:45:34.947458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.947435    5162 scope.go:122] "RemoveContainer" containerID="c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8"
Nov 03 22:45:34.947781 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:45:34.947750    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8\": container with ID starting with c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8 not found: ID does not exist" containerID="c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8"
Nov 03 22:45:34.947888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.947779    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8"} err="failed to get container status \"c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8\": rpc error: code = NotFound desc = could not find container \"c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8\": container with ID starting with c4751772bc2b65a9dd5e58e22e46e9f1ce79b70241914b5f7c69c53e277a46a8 not found: ID does not exist"
Nov 03 22:45:34.991783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.991755    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-plugins-config-volume\") pod \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") "
Nov 03 22:45:34.991985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.991904    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/844839e5-c3c6-447e-af61-ccf4eb0edda5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844839e5-c3c6-447e-af61-ccf4eb0edda5-plugins-config-volume\") pod \"844839e5-c3c6-447e-af61-ccf4eb0edda5\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") "
Nov 03 22:45:34.991985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.991951    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/844839e5-c3c6-447e-af61-ccf4eb0edda5-kube-api-access-f4zxk\" (UniqueName: \"kubernetes.io/projected/844839e5-c3c6-447e-af61-ccf4eb0edda5-kube-api-access-f4zxk\") pod \"844839e5-c3c6-447e-af61-ccf4eb0edda5\" (UID: \"844839e5-c3c6-447e-af61-ccf4eb0edda5\") "
Nov 03 22:45:34.992344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.992255    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-kube-api-access-b2x8h\" (UniqueName: \"kubernetes.io/projected/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-kube-api-access-b2x8h\") pod \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\" (UID: \"b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e\") "
Nov 03 22:45:34.992480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.992147    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-plugins-config-volume" pod "b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" (UID: "b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:45:34.992480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.992365    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/844839e5-c3c6-447e-af61-ccf4eb0edda5-plugins-config-volume" pod "844839e5-c3c6-447e-af61-ccf4eb0edda5" (UID: "844839e5-c3c6-447e-af61-ccf4eb0edda5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:45:34.992962 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.992940    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844839e5-c3c6-447e-af61-ccf4eb0edda5-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:45:34.993059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.992968    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:45:34.994351 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.994330    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844839e5-c3c6-447e-af61-ccf4eb0edda5-kube-api-access-f4zxk" pod "844839e5-c3c6-447e-af61-ccf4eb0edda5" (UID: "844839e5-c3c6-447e-af61-ccf4eb0edda5"). InnerVolumeSpecName "kube-api-access-f4zxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:45:34.994448 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:34.994374    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-kube-api-access-b2x8h" pod "b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" (UID: "b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e"). InnerVolumeSpecName "kube-api-access-b2x8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:45:35.095057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.094979    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4zxk\" (UniqueName: \"kubernetes.io/projected/844839e5-c3c6-447e-af61-ccf4eb0edda5-kube-api-access-f4zxk\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:45:35.095057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.095016    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2x8h\" (UniqueName: \"kubernetes.io/projected/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e-kube-api-access-b2x8h\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:45:35.355438 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.355314    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"]
Nov 03 22:45:35.362709 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.362686    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h"]
Nov 03 22:45:35.476470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.476441    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"]
Nov 03 22:45:35.480195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.480174    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q"]
Nov 03 22:45:35.582544 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.582506    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h" podUID="b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.80:50052\" within 1s: context deadline exceeded"
Nov 03 22:45:35.881878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:35.881845    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:36.555691 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.554779    5162 patch_prober.go:29] interesting pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": read tcp 10.134.0.2:45700->10.134.0.82:15021: read: connection reset by peer" start-of-body=
Nov 03 22:45:36.555691 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.554852    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": read tcp 10.134.0.2:45700->10.134.0.82:15021: read: connection reset by peer"
Nov 03 22:45:36.855548 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.855469    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="844839e5-c3c6-447e-af61-ccf4eb0edda5" path="/var/lib/kubelet/pods/844839e5-c3c6-447e-af61-ccf4eb0edda5/volumes"
Nov 03 22:45:36.855901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.855893    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" path="/var/lib/kubelet/pods/b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e/volumes"
Nov 03 22:45:36.946191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.946163    5162 generic.go:356] "Generic (PLEG): container finished" podID="cf015953-06ee-4239-b7e9-ed60c868e307" containerID="5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a" exitCode=0
Nov 03 22:45:36.946349 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.946230    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" event={"ID":"cf015953-06ee-4239-b7e9-ed60c868e307","Type":"ContainerDied","Data":"5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a"}
Nov 03 22:45:36.946647 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.946633    5162 scope.go:122] "RemoveContainer" containerID="5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a"
Nov 03 22:45:36.948430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.948362    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:45:36.948528 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.948490    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:45:36.948579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:36.948544    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:45:37.973970 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:37.973930    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" event={"ID":"cf015953-06ee-4239-b7e9-ed60c868e307","Type":"ContainerStarted","Data":"30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b"}
Nov 03 22:45:38.612557 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:38.612528    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:45:38.612753 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:38.612690    5162 patch_prober.go:29] interesting pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 03 22:45:38.612836 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:38.612743    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 03 22:45:39.612330 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:39.612300    5162 patch_prober.go:29] interesting pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 03 22:45:39.612764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:39.612349    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 03 22:45:40.091742 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:45:40.091707    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:45:40.612218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:40.612185    5162 patch_prober.go:29] interesting pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 03 22:45:40.612443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:40.612234    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 03 22:45:41.230583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:41.230551    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:41.612503 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:41.612422    5162 patch_prober.go:29] interesting pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 03 22:45:41.612503 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:41.612485    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 03 22:45:42.416440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:42.413213    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:42.612495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:42.612461    5162 patch_prober.go:29] interesting pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 03 22:45:42.612672 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:42.612529    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 03 22:45:42.633851 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:42.633473    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:43.386177 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:43.386120    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:43.645533 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:43.645443    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:43.655735 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:43.655705    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:45:43.656977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:43.656957    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:45:43.658814 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:43.658797    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:45:46.575057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:46.575009    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:50.410692 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:50.410651    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:51.622494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:51.622456    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:45:56.521797 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:45:56.521757    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:00.169804 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:00.169767    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:01.428469 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:01.428429    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:04.282994 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:04.282958    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:06.436613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:06.436576    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:10.086437 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:46:10.086323    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:46:10.454554 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:10.454518    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:11.043323 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.043285    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:11.069201 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.069154    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"]
Nov 03 22:46:11.069727 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.069661    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv" podUID="23923481-cfbe-4e62-8315-e0012658a189" containerName="maas-api" containerID="cri-o://d77ee6add35304f2c608e8ff5268e660b20046791a377ac75c14ac783e4d3b5e" gracePeriod=30
Nov 03 22:46:11.260679 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.260630    5162 generic.go:356] "Generic (PLEG): container finished" podID="23923481-cfbe-4e62-8315-e0012658a189" containerID="d77ee6add35304f2c608e8ff5268e660b20046791a377ac75c14ac783e4d3b5e" exitCode=0
Nov 03 22:46:11.261092 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.260788    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv" event={"ID":"23923481-cfbe-4e62-8315-e0012658a189","Type":"ContainerDied","Data":"d77ee6add35304f2c608e8ff5268e660b20046791a377ac75c14ac783e4d3b5e"}
Nov 03 22:46:11.447115 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.447069    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:46:11.531479 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.531384    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23923481-cfbe-4e62-8315-e0012658a189-kube-api-access-glsqm\" (UniqueName: \"kubernetes.io/projected/23923481-cfbe-4e62-8315-e0012658a189-kube-api-access-glsqm\") pod \"23923481-cfbe-4e62-8315-e0012658a189\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") "
Nov 03 22:46:11.531801 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.531773    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-maas-api-tls\") pod \"23923481-cfbe-4e62-8315-e0012658a189\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") "
Nov 03 22:46:11.531932 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.531831    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-metrics-certs\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-metrics-certs\") pod \"23923481-cfbe-4e62-8315-e0012658a189\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") "
Nov 03 22:46:11.532084 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.532063    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23923481-cfbe-4e62-8315-e0012658a189-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23923481-cfbe-4e62-8315-e0012658a189-openshift-service-ca\") pod \"23923481-cfbe-4e62-8315-e0012658a189\" (UID: \"23923481-cfbe-4e62-8315-e0012658a189\") "
Nov 03 22:46:11.533053 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.533029    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23923481-cfbe-4e62-8315-e0012658a189-openshift-service-ca" pod "23923481-cfbe-4e62-8315-e0012658a189" (UID: "23923481-cfbe-4e62-8315-e0012658a189"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:46:11.538517 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.538491    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-metrics-certs" pod "23923481-cfbe-4e62-8315-e0012658a189" (UID: "23923481-cfbe-4e62-8315-e0012658a189"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:46:11.538900 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.538882    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-maas-api-tls" pod "23923481-cfbe-4e62-8315-e0012658a189" (UID: "23923481-cfbe-4e62-8315-e0012658a189"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:46:11.543132 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.543078    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23923481-cfbe-4e62-8315-e0012658a189-kube-api-access-glsqm" pod "23923481-cfbe-4e62-8315-e0012658a189" (UID: "23923481-cfbe-4e62-8315-e0012658a189"). InnerVolumeSpecName "kube-api-access-glsqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:46:11.634006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.633974    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glsqm\" (UniqueName: \"kubernetes.io/projected/23923481-cfbe-4e62-8315-e0012658a189-kube-api-access-glsqm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:11.634006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.634003    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:11.634209 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.634014    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23923481-cfbe-4e62-8315-e0012658a189-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:11.634209 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:11.634023    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23923481-cfbe-4e62-8315-e0012658a189-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:12.269038 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:12.269007    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"
Nov 03 22:46:12.269563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:12.269038    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv" event={"ID":"23923481-cfbe-4e62-8315-e0012658a189","Type":"ContainerDied","Data":"dced73ef82579affb8b54388a43b10a6be3e8663d7dc8cf10b14f978948b3199"}
Nov 03 22:46:12.269563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:12.269107    5162 scope.go:122] "RemoveContainer" containerID="d77ee6add35304f2c608e8ff5268e660b20046791a377ac75c14ac783e4d3b5e"
Nov 03 22:46:12.441181 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:12.441147    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"]
Nov 03 22:46:12.493685 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:12.493653    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb95c832-6646967857-7ckgv"]
Nov 03 22:46:12.775768 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:12.775732    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:12.856159 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:12.856132    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23923481-cfbe-4e62-8315-e0012658a189" path="/var/lib/kubelet/pods/23923481-cfbe-4e62-8315-e0012658a189/volumes"
Nov 03 22:46:13.300696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:13.300662    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"]
Nov 03 22:46:13.352032 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:13.352003    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bb95c832-qfj8m"]
Nov 03 22:46:14.856192 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:14.856163    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" path="/var/lib/kubelet/pods/8b1e8790-7bb5-4b0d-81ad-5018ffcad95f/volumes"
Nov 03 22:46:15.531415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:15.524109    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:15.731202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:15.731165    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:17.488074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.488033    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:17.970428 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.970384    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"]
Nov 03 22:46:17.971052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971039    5162 cpu_manager.go:414] "containerMap: removing container" podUID="b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" containerName="payload-processing"
Nov 03 22:46:17.971098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971053    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" containerName="payload-processing"
Nov 03 22:46:17.971098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971067    5162 cpu_manager.go:414] "containerMap: removing container" podUID="844839e5-c3c6-447e-af61-ccf4eb0edda5" containerName="payload-pre-processing"
Nov 03 22:46:17.971098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971072    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="844839e5-c3c6-447e-af61-ccf4eb0edda5" containerName="payload-pre-processing"
Nov 03 22:46:17.971098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971080    5162 cpu_manager.go:414] "containerMap: removing container" podUID="8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" containerName="revoke-keys"
Nov 03 22:46:17.971098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971085    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" containerName="revoke-keys"
Nov 03 22:46:17.971257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971124    5162 cpu_manager.go:414] "containerMap: removing container" podUID="23923481-cfbe-4e62-8315-e0012658a189" containerName="maas-api"
Nov 03 22:46:17.971257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971130    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23923481-cfbe-4e62-8315-e0012658a189" containerName="maas-api"
Nov 03 22:46:17.971257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971229    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="23923481-cfbe-4e62-8315-e0012658a189" containerName="maas-api"
Nov 03 22:46:17.971257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971241    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3dc7acf-7733-47cc-9cbc-8a94b2a56b1e" containerName="payload-processing"
Nov 03 22:46:17.971257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971247    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="844839e5-c3c6-447e-af61-ccf4eb0edda5" containerName="payload-pre-processing"
Nov 03 22:46:17.971257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.971254    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b1e8790-7bb5-4b0d-81ad-5018ffcad95f" containerName="revoke-keys"
Nov 03 22:46:17.976275 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:17.976252    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.039486 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.036771    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"]
Nov 03 22:46:18.096520 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.096490    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d18a5b05-openshift-default-dockercfg-zksq2\""
Nov 03 22:46:18.098752 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.098721    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-certs\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.098985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.098966    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdgxf\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-kube-api-access-zdgxf\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.099109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.099093    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-data\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.099233 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.099219    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-token\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.099406 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.099373    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-envoy\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.099603 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.099587    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-credential-socket\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.099785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.099770    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-socket\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.099957 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.099942    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da0a843f-e5e3-43b7-93fd-f66a407d066e-istiod-ca-cert\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.100079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.100065    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-podinfo\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.165946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.158572    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:18.202059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.202017    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-certs\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.202459 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.202431    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdgxf\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-kube-api-access-zdgxf\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.202681 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.202662    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-data\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.202867 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.202849    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-token\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203037    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-envoy\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203220    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-credential-socket\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203449 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203431    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-socket\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203296    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-certs\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203446    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-data\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203636    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da0a843f-e5e3-43b7-93fd-f66a407d066e-istiod-ca-cert\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203677    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-credential-socket\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.203800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203685    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-podinfo\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.204110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.203845    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-socket\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.204841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.204705    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da0a843f-e5e3-43b7-93fd-f66a407d066e-istiod-ca-cert\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.224806 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.224682    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-podinfo\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.228117 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.228064    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-envoy\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.274564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.274532    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdgxf\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-kube-api-access-zdgxf\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.289185 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.289122    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-token\") pod \"e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") " pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.301332 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.301297    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:18.369355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.366138    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:18.616939 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.616850    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:18.743550 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:46:18.743512    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0a843f_e5e3_43b7_93fd_f66a407d066e.slice/crio-112846e92d5d55a8be8007f2a0bb63a5b487440d1c26c872a53ed0c3d29e007d WatchSource:0}: Error finding container 112846e92d5d55a8be8007f2a0bb63a5b487440d1c26c872a53ed0c3d29e007d: Status 404 returned error can't find the container with id 112846e92d5d55a8be8007f2a0bb63a5b487440d1c26c872a53ed0c3d29e007d
Nov 03 22:46:18.746249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.746213    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:46:18.746344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.746279    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:46:18.746344 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.746309    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:46:18.748106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.748083    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"]
Nov 03 22:46:18.825496 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:18.824828    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:19.037097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:19.037033    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:19.245019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:19.244981    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:19.325775 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:19.325680    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" event={"ID":"da0a843f-e5e3-43b7-93fd-f66a407d066e","Type":"ContainerStarted","Data":"12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1"}
Nov 03 22:46:19.325775 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:19.325763    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" event={"ID":"da0a843f-e5e3-43b7-93fd-f66a407d066e","Type":"ContainerStarted","Data":"112846e92d5d55a8be8007f2a0bb63a5b487440d1c26c872a53ed0c3d29e007d"}
Nov 03 22:46:19.450259 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:19.450209    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podStartSLOduration=2.450195442 podStartE2EDuration="2.450195442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:19.448626108 +0000 UTC m=+1382.713326120" watchObservedRunningTime="2026-11-03 22:46:19.450195442 +0000 UTC m=+1382.714895452"
Nov 03 22:46:20.302300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:20.302258    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:20.307824 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:20.307799    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:20.333143 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:20.333120    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:20.333888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:20.333867    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:46:23.055990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.055957    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"]
Nov 03 22:46:23.056432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.056170    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj" podUID="7375ed63-6334-4fa1-a2ca-a3db7444ae36" containerName="maas-api" containerID="cri-o://9ef2295c00dc632224b4c5253a32deb0a907b07719ebc6db59aeaf55114db416" gracePeriod=30
Nov 03 22:46:23.214109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.214056    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:23.365872 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.365840    5162 generic.go:356] "Generic (PLEG): container finished" podID="7375ed63-6334-4fa1-a2ca-a3db7444ae36" containerID="9ef2295c00dc632224b4c5253a32deb0a907b07719ebc6db59aeaf55114db416" exitCode=0
Nov 03 22:46:23.366065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.365996    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj" event={"ID":"7375ed63-6334-4fa1-a2ca-a3db7444ae36","Type":"ContainerDied","Data":"9ef2295c00dc632224b4c5253a32deb0a907b07719ebc6db59aeaf55114db416"}
Nov 03 22:46:23.366065 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.366049    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj" event={"ID":"7375ed63-6334-4fa1-a2ca-a3db7444ae36","Type":"ContainerDied","Data":"e971a4b0b9b46779652f29f93b0f404169741f528d3c7df5a5b32090b576034e"}
Nov 03 22:46:23.366170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.366077    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e971a4b0b9b46779652f29f93b0f404169741f528d3c7df5a5b32090b576034e"
Nov 03 22:46:23.380172 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.380139    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:46:23.468694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.468359    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:23.511817 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.511787    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7375ed63-6334-4fa1-a2ca-a3db7444ae36-kube-api-access-jtqh8\" (UniqueName: \"kubernetes.io/projected/7375ed63-6334-4fa1-a2ca-a3db7444ae36-kube-api-access-jtqh8\") pod \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") "
Nov 03 22:46:23.512076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.512055    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls\") pod \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") "
Nov 03 22:46:23.521948 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.521915    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7375ed63-6334-4fa1-a2ca-a3db7444ae36-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7375ed63-6334-4fa1-a2ca-a3db7444ae36-openshift-service-ca\") pod \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") "
Nov 03 22:46:23.522332 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.522311    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-metrics-certs\") pod \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\" (UID: \"7375ed63-6334-4fa1-a2ca-a3db7444ae36\") "
Nov 03 22:46:23.565666 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.565603    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7375ed63-6334-4fa1-a2ca-a3db7444ae36-openshift-service-ca" pod "7375ed63-6334-4fa1-a2ca-a3db7444ae36" (UID: "7375ed63-6334-4fa1-a2ca-a3db7444ae36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:46:23.591476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.573508    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls" pod "7375ed63-6334-4fa1-a2ca-a3db7444ae36" (UID: "7375ed63-6334-4fa1-a2ca-a3db7444ae36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:46:23.591476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.590588    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7375ed63-6334-4fa1-a2ca-a3db7444ae36-kube-api-access-jtqh8" pod "7375ed63-6334-4fa1-a2ca-a3db7444ae36" (UID: "7375ed63-6334-4fa1-a2ca-a3db7444ae36"). InnerVolumeSpecName "kube-api-access-jtqh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:46:23.592310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.592264    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-metrics-certs" pod "7375ed63-6334-4fa1-a2ca-a3db7444ae36" (UID: "7375ed63-6334-4fa1-a2ca-a3db7444ae36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:46:23.648857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.648824    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:23.648857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.648851    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtqh8\" (UniqueName: \"kubernetes.io/projected/7375ed63-6334-4fa1-a2ca-a3db7444ae36-kube-api-access-jtqh8\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:23.648857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.648864    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7375ed63-6334-4fa1-a2ca-a3db7444ae36-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:23.649087 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.648874    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7375ed63-6334-4fa1-a2ca-a3db7444ae36-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:46:23.673556 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.673517    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:23.925313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.925276    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"]
Nov 03 22:46:23.926073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.926045    5162 cpu_manager.go:414] "containerMap: removing container" podUID="7375ed63-6334-4fa1-a2ca-a3db7444ae36" containerName="maas-api"
Nov 03 22:46:23.926073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.926070    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7375ed63-6334-4fa1-a2ca-a3db7444ae36" containerName="maas-api"
Nov 03 22:46:23.926265 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.926255    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="7375ed63-6334-4fa1-a2ca-a3db7444ae36" containerName="maas-api"
Nov 03 22:46:23.930176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.930157    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:23.985122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.985092    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-e7a91ca4\""
Nov 03 22:46:23.985289 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.985207    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-e7a91ca4-dockercfg-9j6fq\""
Nov 03 22:46:23.991708 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:23.991678    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"]
Nov 03 22:46:24.166453 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.162677    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d89bb0-a280-440b-8def-3a640b5e2320-plugins-config-volume\") pod \"payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") " pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:24.166453 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.162902    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4cdx\" (UniqueName: \"kubernetes.io/projected/43d89bb0-a280-440b-8def-3a640b5e2320-kube-api-access-l4cdx\") pod \"payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") " pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:24.264308 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.264224    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d89bb0-a280-440b-8def-3a640b5e2320-plugins-config-volume\") pod \"payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") " pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:24.264514 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.264457    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4cdx\" (UniqueName: \"kubernetes.io/projected/43d89bb0-a280-440b-8def-3a640b5e2320-kube-api-access-l4cdx\") pod \"payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") " pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:24.265044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.265019    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d89bb0-a280-440b-8def-3a640b5e2320-plugins-config-volume\") pod \"payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") " pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:24.343346 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.343316    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4cdx\" (UniqueName: \"kubernetes.io/projected/43d89bb0-a280-440b-8def-3a640b5e2320-kube-api-access-l4cdx\") pod \"payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") " pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:24.371568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.371537    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:24.372914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.372894    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"
Nov 03 22:46:24.544431 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.544328    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:24.556925 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.556899    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"]
Nov 03 22:46:24.587614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.587584    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:24.607881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.607854    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-027a6fbe-84846ff457-6k7wj"]
Nov 03 22:46:24.855728 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.855642    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7375ed63-6334-4fa1-a2ca-a3db7444ae36" path="/var/lib/kubelet/pods/7375ed63-6334-4fa1-a2ca-a3db7444ae36/volumes"
Nov 03 22:46:24.926668 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:46:24.926637    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d89bb0_a280_440b_8def_3a640b5e2320.slice/crio-e9a22fdd2c38b6d32d0fab7913f484fe7710a192d545d9197c2804884deb4cff WatchSource:0}: Error finding container e9a22fdd2c38b6d32d0fab7913f484fe7710a192d545d9197c2804884deb4cff: Status 404 returned error can't find the container with id e9a22fdd2c38b6d32d0fab7913f484fe7710a192d545d9197c2804884deb4cff
Nov 03 22:46:24.931692 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:24.931668    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"]
Nov 03 22:46:25.159697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.159607    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:25.362820 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.362781    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:25.381643 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.381607    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x" event={"ID":"43d89bb0-a280-440b-8def-3a640b5e2320","Type":"ContainerStarted","Data":"dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5"}
Nov 03 22:46:25.381785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.381661    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x" event={"ID":"43d89bb0-a280-440b-8def-3a640b5e2320","Type":"ContainerStarted","Data":"e9a22fdd2c38b6d32d0fab7913f484fe7710a192d545d9197c2804884deb4cff"}
Nov 03 22:46:25.381853 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.381826    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:25.504749 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.504686    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x" podStartSLOduration=2.504667394 podStartE2EDuration="2.504667394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:25.500705476 +0000 UTC m=+1388.765405487" watchObservedRunningTime="2026-11-03 22:46:25.504667394 +0000 UTC m=+1388.769367405"
Nov 03 22:46:25.564371 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.564335    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:25.837158 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:25.837063    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:26.817696 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:26.817657    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:27.019499 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:27.019468    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:27.713733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:27.713700    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:27.964830 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:27.964736    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:30.645359 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:30.645319    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:33.769073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:33.769035    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:33.968250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:33.968204    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:35.349320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:35.349276    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:35.554068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:35.553653    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:36.018598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:36.018559    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:36.393540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:36.393462    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:46:40.086756 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:46:40.086716    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:46:40.922444 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:40.922353    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:43.700174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.700141    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"]
Nov 03 22:46:43.704658 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.704639    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.747295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.747269    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.747446 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.747305    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3268422-a2de-48be-ba19-1860bead948c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.747446 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.747434    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.747547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.747486    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.747601 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.747588    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.747679 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.747667    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpn25\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-kube-api-access-wpn25\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.747719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.747708    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.758864 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.758840    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 22:46:43.758964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.758887    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 22:46:43.758964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.758940    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 22:46:43.759118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.759106    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8qdxr\""
Nov 03 22:46:43.814768 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.814740    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"]
Nov 03 22:46:43.849496 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.849467    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.849637 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.849559    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpn25\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-kube-api-access-wpn25\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.849637 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.849586    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.849723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.849696    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.849760 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.849727    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3268422-a2de-48be-ba19-1860bead948c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.849801 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.849783    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.849872 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.849855    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.850245 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.850222    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.852059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.852034    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3268422-a2de-48be-ba19-1860bead948c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.852166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.852152    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.852502 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.852485    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.852581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.852499    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.912191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.912156    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:43.912320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:43.912310    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpn25\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-kube-api-access-wpn25\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xkzbs\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:44.019961 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:44.019895    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:44.453754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:44.453717    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"]
Nov 03 22:46:44.482838 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:46:44.482810    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3268422_a2de_48be_ba19_1860bead948c.slice/crio-66a2af47e81463b09d77e18ff6281e32eb62a23274ba6088c4da39c2ea3c7fdd WatchSource:0}: Error finding container 66a2af47e81463b09d77e18ff6281e32eb62a23274ba6088c4da39c2ea3c7fdd: Status 404 returned error can't find the container with id 66a2af47e81463b09d77e18ff6281e32eb62a23274ba6088c4da39c2ea3c7fdd
Nov 03 22:46:44.569714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:44.569682    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs" event={"ID":"c3268422-a2de-48be-ba19-1860bead948c","Type":"ContainerStarted","Data":"66a2af47e81463b09d77e18ff6281e32eb62a23274ba6088c4da39c2ea3c7fdd"}
Nov 03 22:46:45.823312 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:45.823276    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:48.283115 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:48.283078    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:46:48.283442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:48.283146    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:46:48.606207 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:48.606108    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs" event={"ID":"c3268422-a2de-48be-ba19-1860bead948c","Type":"ContainerStarted","Data":"af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143"}
Nov 03 22:46:48.606347 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:48.606211    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:48.836254 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:48.836185    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs" podStartSLOduration=2.039215449 podStartE2EDuration="5.836172664s" totalImagesPullingTime="3.796957215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:48.781589808 +0000 UTC m=+1412.046289818" watchObservedRunningTime="2026-11-03 22:46:48.836172664 +0000 UTC m=+1412.100872674"
Nov 03 22:46:49.620466 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:49.620431    5162 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:46:49.621013 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:49.620987    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs" podUID="c3268422-a2de-48be-ba19-1860bead948c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:46:50.720613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:50.720559    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:50.923037 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:50.923000    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:52.619324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:52.619294    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 22:46:55.632374 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:55.632322    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:46:59.719072 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:59.719040    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"]
Nov 03 22:46:59.719618 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:59.719236    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc" podUID="f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" containerName="maas-api" containerID="cri-o://ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04" gracePeriod=30
Nov 03 22:46:59.983062 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:46:59.982687    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:00.061609 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.061580    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:47:00.103646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.103611    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-openshift-service-ca\") pod \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") "
Nov 03 22:47:00.103808 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.103719    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls\") pod \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") "
Nov 03 22:47:00.103850 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.103840    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-metrics-certs\") pod \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") "
Nov 03 22:47:00.103965 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.103936    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-openshift-service-ca" pod "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" (UID: "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:47:00.103965 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.103952    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-kube-api-access-pswzx\" (UniqueName: \"kubernetes.io/projected/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-kube-api-access-pswzx\") pod \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\" (UID: \"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00\") "
Nov 03 22:47:00.104653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.104630    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:00.105944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.105922    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls" pod "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" (UID: "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:47:00.106074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.106047    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-kube-api-access-pswzx" pod "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" (UID: "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00"). InnerVolumeSpecName "kube-api-access-pswzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:47:00.106163 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.106112    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-metrics-certs" pod "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" (UID: "f0a66bd9-1980-42b2-8c2f-3fe2188a5f00"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:47:00.252320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.252225    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:00.252320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.252265    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pswzx\" (UniqueName: \"kubernetes.io/projected/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-kube-api-access-pswzx\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:00.252320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.252282    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:00.596504 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.596415    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:00.702486 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.702447    5162 generic.go:356] "Generic (PLEG): container finished" podID="f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" containerID="ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04" exitCode=0
Nov 03 22:47:00.702703 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.702538    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc" event={"ID":"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00","Type":"ContainerDied","Data":"ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04"}
Nov 03 22:47:00.703969 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.702625    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc" event={"ID":"f0a66bd9-1980-42b2-8c2f-3fe2188a5f00","Type":"ContainerDied","Data":"84da3e19d1b905d412842666057dce0a0d9393cbe34903b06dfb8dd5cfe785d8"}
Nov 03 22:47:00.703969 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.702800    5162 scope.go:122] "RemoveContainer" containerID="ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04"
Nov 03 22:47:00.703969 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.702895    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"
Nov 03 22:47:00.740416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.737529    5162 scope.go:122] "RemoveContainer" containerID="ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04"
Nov 03 22:47:00.755417 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:47:00.753032    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04\": container with ID starting with ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04 not found: ID does not exist" containerID="ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04"
Nov 03 22:47:00.755417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.753080    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04"} err="failed to get container status \"ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04\": rpc error: code = NotFound desc = could not find container \"ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04\": container with ID starting with ccaccdac26a31a2d91cbd5841848f4202deae641b42d9a53740266f8cd11fd04 not found: ID does not exist"
Nov 03 22:47:00.920104 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.920077    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"]
Nov 03 22:47:00.923940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:00.923914    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c082ab23-797fc6564-h5flc"]
Nov 03 22:47:01.100458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:01.100425    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:01.485700 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:01.485658    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:01.776463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:01.776366    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:02.726694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:02.726663    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:02.856973 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:02.856942    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" path="/var/lib/kubelet/pods/f0a66bd9-1980-42b2-8c2f-3fe2188a5f00/volumes"
Nov 03 22:47:03.064518 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:03.064418    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:03.272309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:03.272275    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:03.392846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:03.392759    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"]
Nov 03 22:47:03.393044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:03.393021    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy" containerID="cri-o://30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b" gracePeriod=30
Nov 03 22:47:03.524493 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:03.524458    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:03.735733 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:03.735698    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:04.400059 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:04.400022    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:04.613422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:04.613356    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:05.292138 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:05.292106    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:05.733149 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:05.733118    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:06.008601 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:06.008512    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:06.307738 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:06.307638    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:08.275173 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.275140    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:08.487380 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.487343    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:08.697382 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.697359    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:47:08.701710 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.701683    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:08.797358 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.797329    5162 generic.go:356] "Generic (PLEG): container finished" podID="cf015953-06ee-4239-b7e9-ed60c868e307" containerID="30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b" exitCode=0
Nov 03 22:47:08.797521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.797368    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" event={"ID":"cf015953-06ee-4239-b7e9-ed60c868e307","Type":"ContainerDied","Data":"30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b"}
Nov 03 22:47:08.797521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.797378    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"
Nov 03 22:47:08.797521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.797427    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9" event={"ID":"cf015953-06ee-4239-b7e9-ed60c868e307","Type":"ContainerDied","Data":"0df219d8ff244cbf7a9e533b3377e8e8091677eb1f52c1c315af3b08f3a38f04"}
Nov 03 22:47:08.797521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.797447    5162 scope.go:122] "RemoveContainer" containerID="30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b"
Nov 03 22:47:08.812093 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.812075    5162 scope.go:122] "RemoveContainer" containerID="5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a"
Nov 03 22:47:08.818673 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.818650    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-kube-api-access-5t6s9\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-kube-api-access-5t6s9\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.818823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.818804    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-certs\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.818916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.818900    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf015953-06ee-4239-b7e9-ed60c868e307-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf015953-06ee-4239-b7e9-ed60c868e307-istio-podinfo\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.818973 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.818955    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-credential-socket\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.819102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819075    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-certs" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:08.819194 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819170    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf015953-06ee-4239-b7e9-ed60c868e307-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf015953-06ee-4239-b7e9-ed60c868e307-istiod-ca-cert\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.819249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819217    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-socket\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.819288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819247    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-envoy\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.819341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819288    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-data\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.819341 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819212    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-credential-socket" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819367    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-istio-token\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-istio-token\") pod \"cf015953-06ee-4239-b7e9-ed60c868e307\" (UID: \"cf015953-06ee-4239-b7e9-ed60c868e307\") "
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819410    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-socket" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819468    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf015953-06ee-4239-b7e9-ed60c868e307-istiod-ca-cert" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.819626    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-data" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.820015    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf015953-06ee-4239-b7e9-ed60c868e307-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.820038    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.820054    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.820064    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.820184 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.820073    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.821322 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.821300    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-envoy" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:08.821457 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.821440    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf015953-06ee-4239-b7e9-ed60c868e307-istio-podinfo" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:47:08.821532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.821521    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-kube-api-access-5t6s9" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "kube-api-access-5t6s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:47:08.821859 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.821834    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-istio-token" pod "cf015953-06ee-4239-b7e9-ed60c868e307" (UID: "cf015953-06ee-4239-b7e9-ed60c868e307"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:47:08.823217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.823200    5162 scope.go:122] "RemoveContainer" containerID="30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b"
Nov 03 22:47:08.823568 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:47:08.823543    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b\": container with ID starting with 30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b not found: ID does not exist" containerID="30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b"
Nov 03 22:47:08.823634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.823580    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b"} err="failed to get container status \"30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b\": rpc error: code = NotFound desc = could not find container \"30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b\": container with ID starting with 30682fbe4543737f549867997384b691afb3deb8f0a03b9451d03be3d706a36b not found: ID does not exist"
Nov 03 22:47:08.823634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.823605    5162 scope.go:122] "RemoveContainer" containerID="5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a"
Nov 03 22:47:08.823899 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:47:08.823877    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a\": container with ID starting with 5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a not found: ID does not exist" containerID="5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a"
Nov 03 22:47:08.823946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.823907    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a"} err="failed to get container status \"5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a\": rpc error: code = NotFound desc = could not find container \"5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a\": container with ID starting with 5286dc4b013048781dc9be855246f0e17a9e2949459af8359ddf5508c08ebc7a not found: ID does not exist"
Nov 03 22:47:08.921683 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.921652    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf015953-06ee-4239-b7e9-ed60c868e307-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.921683 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.921682    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.921933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.921695    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t6s9\" (UniqueName: \"kubernetes.io/projected/cf015953-06ee-4239-b7e9-ed60c868e307-kube-api-access-5t6s9\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:08.921933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:08.921710    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf015953-06ee-4239-b7e9-ed60c868e307-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:09.220927 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:09.220899    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"]
Nov 03 22:47:09.224575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:09.224556    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9"]
Nov 03 22:47:10.085008 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:47:10.084971    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:47:10.771803 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:10.771759    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:10.856701 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:10.856671    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" path="/var/lib/kubelet/pods/cf015953-06ee-4239-b7e9-ed60c868e307/volumes"
Nov 03 22:47:11.020013 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:11.019977    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:13.521373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:13.521338    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:13.725071 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:13.725037    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:13.938946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:13.938900    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:19.026195 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:19.026155    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:19.246103 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:19.246069    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:22.495915 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:22.495841    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:22.925828 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:22.925797    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:23.589070 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:23.588989    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:24.131252 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:24.131221    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:24.690766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:24.690736    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:27.649828 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:27.649790    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:27.943990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:27.943953    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:28.177440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:28.177404    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:28.987675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:28.987644    5162 generic.go:356] "Generic (PLEG): container finished" podID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerID="ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8" exitCode=0
Nov 03 22:47:28.988134 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:28.987719    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" event={"ID":"2c9e36e5-5009-4773-9b24-42c4d14fa329","Type":"ContainerDied","Data":"ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8"}
Nov 03 22:47:28.988209 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:28.988171    5162 scope.go:122] "RemoveContainer" containerID="ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8"
Nov 03 22:47:28.989894 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:28.989863    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:47:28.989986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:28.989930    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:47:28.989986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:28.989967    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:47:29.998326 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:29.998293    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" event={"ID":"2c9e36e5-5009-4773-9b24-42c4d14fa329","Type":"ContainerStarted","Data":"d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef"}
Nov 03 22:47:30.474413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:30.473437    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:30.490796 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:30.482766    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:47:30.490796 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:30.489778    5162 patch_prober.go:29] interesting pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:47:30.490796 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:30.489842    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:47:31.301581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:31.301116    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:31.480058 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:31.480023    5162 patch_prober.go:29] interesting pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:47:31.480259 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:31.480086    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:47:31.506837 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:31.506800    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:32.480203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:32.480169    5162 patch_prober.go:29] interesting pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:47:32.480742 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:32.480216    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:47:32.762472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:32.761488    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:47:32.967533 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:32.967502    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:33.482623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:33.480600    5162 patch_prober.go:29] interesting pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:47:33.482623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:33.480664    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:47:34.480610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:34.480578    5162 patch_prober.go:29] interesting pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:47:34.480794 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:34.480630    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:47:35.463129 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:35.463092    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:35.479790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:35.479756    5162 patch_prober.go:29] interesting pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:47:35.479974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:35.479822    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:47:36.484169 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:36.484139    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:47:36.485073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:36.485057    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 22:47:38.962095 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:38.961987    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:39.270411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:39.270316    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:40.085826 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:47:40.085786    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:47:40.321937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:40.320623    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:41.422926 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:41.422882    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:45.191765 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:45.191734    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:45.440500 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:45.440466    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:47.577305 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:47.577273    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:50.011068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:50.011032    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:51.139681 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:51.139628    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:51.172716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:51.172686    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"]
Nov 03 22:47:51.172881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:51.172864    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy" containerID="cri-o://bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90" gracePeriod=30
Nov 03 22:47:54.810677 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:54.810625    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:47:56.506730 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.506705    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:47:56.639418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639374    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-credential-socket\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.639711 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639450    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-kube-api-access-thd5l\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-kube-api-access-thd5l\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.639711 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639531    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istiod-ca-cert\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.639837 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639726    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-credential-socket" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:56.639837 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639744    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-token\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-token\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.639837 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639778    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-envoy\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.639990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639836    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-socket\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.639990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.639919    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-certs\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.640106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.640012    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-data\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.640106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.640059    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istiod-ca-cert" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:47:56.640106 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.640071    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-podinfo\") pod \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\" (UID: \"8679ecba-3f23-4fc3-8426-d7a29c6bde30\") "
Nov 03 22:47:56.640384 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.640359    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-certs" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:56.640673 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.640650    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-data" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:56.640951 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.640924    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-socket" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:56.641074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.641014    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.641074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.641037    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.641074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.641054    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.641074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.641069    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.643198 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.643152    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-podinfo" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:47:56.643374 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.643342    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-envoy" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:56.643755 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.643720    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-token" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:47:56.645174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.645141    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-kube-api-access-thd5l" pod "8679ecba-3f23-4fc3-8426-d7a29c6bde30" (UID: "8679ecba-3f23-4fc3-8426-d7a29c6bde30"). InnerVolumeSpecName "kube-api-access-thd5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:47:56.749481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.748059    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.749481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.748098    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.749481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.748157    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8679ecba-3f23-4fc3-8426-d7a29c6bde30-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.749481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.748172    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8679ecba-3f23-4fc3-8426-d7a29c6bde30-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:56.749481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:56.748189    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thd5l\" (UniqueName: \"kubernetes.io/projected/8679ecba-3f23-4fc3-8426-d7a29c6bde30-kube-api-access-thd5l\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:47:57.308140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.308094    5162 generic.go:356] "Generic (PLEG): container finished" podID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerID="bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90" exitCode=0
Nov 03 22:47:57.308846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.308429    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" event={"ID":"8679ecba-3f23-4fc3-8426-d7a29c6bde30","Type":"ContainerDied","Data":"bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90"}
Nov 03 22:47:57.309076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.309057    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd" event={"ID":"8679ecba-3f23-4fc3-8426-d7a29c6bde30","Type":"ContainerDied","Data":"71b416e4bb5c948a43cb2b4ebeb8c38d55f3c40db744b188db0b5375f8a00d58"}
Nov 03 22:47:57.309253 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.308536    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"
Nov 03 22:47:57.316353 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.309199    5162 scope.go:122] "RemoveContainer" containerID="bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90"
Nov 03 22:47:57.336969 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.336949    5162 scope.go:122] "RemoveContainer" containerID="bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90"
Nov 03 22:47:57.337422 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:47:57.337382    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90\": container with ID starting with bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90 not found: ID does not exist" containerID="bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90"
Nov 03 22:47:57.337490 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.337430    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90"} err="failed to get container status \"bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90\": rpc error: code = NotFound desc = could not find container \"bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90\": container with ID starting with bceeb9281c4bb893715451b3ea65cfa398f0e76053b32bcc79a83d802ad52b90 not found: ID does not exist"
Nov 03 22:47:57.583799 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.574431    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"]
Nov 03 22:47:57.606187 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:57.606126    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd"]
Nov 03 22:47:58.874524 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:58.874491    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" path="/var/lib/kubelet/pods/8679ecba-3f23-4fc3-8426-d7a29c6bde30/volumes"
Nov 03 22:47:59.721778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:47:59.721745    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:00.898700 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:00.898663    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:04.221122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221088    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"]
Nov 03 22:48:04.221727 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221713    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" containerName="maas-api"
Nov 03 22:48:04.221778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221728    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" containerName="maas-api"
Nov 03 22:48:04.221778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221759    5162 cpu_manager.go:414] "containerMap: removing container" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy"
Nov 03 22:48:04.221778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221765    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy"
Nov 03 22:48:04.221871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221790    5162 cpu_manager.go:414] "containerMap: removing container" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy"
Nov 03 22:48:04.221871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221796    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy"
Nov 03 22:48:04.221871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221805    5162 cpu_manager.go:414] "containerMap: removing container" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy"
Nov 03 22:48:04.221871 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221809    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy"
Nov 03 22:48:04.221993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221895    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="8679ecba-3f23-4fc3-8426-d7a29c6bde30" containerName="istio-proxy"
Nov 03 22:48:04.221993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221904    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy"
Nov 03 22:48:04.221993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.221912    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0a66bd9-1980-42b2-8c2f-3fe2188a5f00" containerName="maas-api"
Nov 03 22:48:04.225317 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.225302    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.279638 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.279609    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"]
Nov 03 22:48:04.326608 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.326573    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-12450a06-serving-cert\""
Nov 03 22:48:04.357438 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.357382    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-maas-api-tls\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.357673 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.357650    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-metrics-certs\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.357781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.357724    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6vdf\" (UniqueName: \"kubernetes.io/projected/132faa2b-51ce-4007-b4f4-dee77876a808-kube-api-access-p6vdf\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.357841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.357783    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/132faa2b-51ce-4007-b4f4-dee77876a808-openshift-service-ca\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.459572 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.459540    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-metrics-certs\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.459766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.459594    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6vdf\" (UniqueName: \"kubernetes.io/projected/132faa2b-51ce-4007-b4f4-dee77876a808-kube-api-access-p6vdf\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.459766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.459627    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/132faa2b-51ce-4007-b4f4-dee77876a808-openshift-service-ca\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.459766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.459739    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-maas-api-tls\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.460533 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.460506    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/132faa2b-51ce-4007-b4f4-dee77876a808-openshift-service-ca\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.462598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.462571    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-metrics-certs\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.466044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.466019    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:04.470121 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.470095    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-maas-api-tls\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.531292 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.531211    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6vdf\" (UniqueName: \"kubernetes.io/projected/132faa2b-51ce-4007-b4f4-dee77876a808-kube-api-access-p6vdf\") pod \"maas-api-e2e-mt-12450a06-68776f84c-xr7x4\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.539950 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.539919    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:04.695246 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.669617    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:04.875236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.875150    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:04.935699 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:48:04.935671    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod132faa2b_51ce_4007_b4f4_dee77876a808.slice/crio-8fbd51b20fec4eeeca2a1d9f14501146eefffb9c91d2cfd21353d352104a09e0 WatchSource:0}: Error finding container 8fbd51b20fec4eeeca2a1d9f14501146eefffb9c91d2cfd21353d352104a09e0: Status 404 returned error can't find the container with id 8fbd51b20fec4eeeca2a1d9f14501146eefffb9c91d2cfd21353d352104a09e0
Nov 03 22:48:04.939827 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:04.939804    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"]
Nov 03 22:48:05.376590 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:05.376552    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4" event={"ID":"132faa2b-51ce-4007-b4f4-dee77876a808","Type":"ContainerStarted","Data":"8fbd51b20fec4eeeca2a1d9f14501146eefffb9c91d2cfd21353d352104a09e0"}
Nov 03 22:48:05.992885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:05.992798    5162 patch_prober.go:29] interesting pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": read tcp 10.134.0.2:32872->10.134.0.89:15021: read: connection reset by peer" start-of-body=
Nov 03 22:48:05.992885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:05.992860    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": read tcp 10.134.0.2:32872->10.134.0.89:15021: read: connection reset by peer"
Nov 03 22:48:06.093650 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.093557    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:06.384933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.384847    5162 generic.go:356] "Generic (PLEG): container finished" podID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerID="12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1" exitCode=0
Nov 03 22:48:06.384933 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.384919    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" event={"ID":"da0a843f-e5e3-43b7-93fd-f66a407d066e","Type":"ContainerDied","Data":"12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1"}
Nov 03 22:48:06.385507 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.385489    5162 scope.go:122] "RemoveContainer" containerID="12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1"
Nov 03 22:48:06.386782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.386754    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4" event={"ID":"132faa2b-51ce-4007-b4f4-dee77876a808","Type":"ContainerStarted","Data":"a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992"}
Nov 03 22:48:06.386900 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.386834    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:06.387342 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.387311    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:48:06.387479 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.387432    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:48:06.387554 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.387488    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:48:06.684524 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:06.684459    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4" podStartSLOduration=1.801889175 podStartE2EDuration="2.684440525s" totalImagesPullingTime="882.55135ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:48:06.68385844 +0000 UTC m=+1489.948558453" watchObservedRunningTime="2026-11-03 22:48:06.684440525 +0000 UTC m=+1489.949140537"
Nov 03 22:48:07.399737 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:07.399701    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" event={"ID":"da0a843f-e5e3-43b7-93fd-f66a407d066e","Type":"ContainerStarted","Data":"aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563"}
Nov 03 22:48:08.301714 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.301674    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:48:08.301920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.301891    5162 patch_prober.go:29] interesting pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:48:08.301974 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.301949    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:48:08.600618 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.600535    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"]
Nov 03 22:48:08.601318 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.601303    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf015953-06ee-4239-b7e9-ed60c868e307" containerName="istio-proxy"
Nov 03 22:48:08.604217 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.604199    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.730349 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730098    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2911c96-53cb-4128-8f79-0975e1b97894-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.730568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730352    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.730568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730409    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.730568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730456    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.730568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730533    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.731060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730675    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.731060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730698    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2911c96-53cb-4128-8f79-0975e1b97894-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.731060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730724    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hgj4\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-kube-api-access-4hgj4\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.731060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.730793    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.733011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.732985    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"]
Nov 03 22:48:08.832698 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.832665    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.832899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.832775    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.832899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.832792    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2911c96-53cb-4128-8f79-0975e1b97894-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.832899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.832807    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hgj4\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-kube-api-access-4hgj4\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.832899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.832852    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.833135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.832904    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2911c96-53cb-4128-8f79-0975e1b97894-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.833135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.832990    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.833135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.833021    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.833135 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.833050    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.833535 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.833509    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.833802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.833775    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.834046 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.834013    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.834155 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.834125    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.834248 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.834229    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2911c96-53cb-4128-8f79-0975e1b97894-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.836022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.835994    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.836829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.836807    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2911c96-53cb-4128-8f79-0975e1b97894-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.895788 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.895713    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.896013 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.895990    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hgj4\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-kube-api-access-4hgj4\") pod \"maas-default-gateway-openshift-default-59f957fdd7-f64cj\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:08.917659 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:08.917628    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:09.302188 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.302157    5162 patch_prober.go:29] interesting pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:48:09.302374 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.302208    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:48:09.374607 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:48:09.374577    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2911c96_53cb_4128_8f79_0975e1b97894.slice/crio-9c2d86935374fe01a6356fb5a01ccd59c0a4d0a6823af567d216b647d50aa793 WatchSource:0}: Error finding container 9c2d86935374fe01a6356fb5a01ccd59c0a4d0a6823af567d216b647d50aa793: Status 404 returned error can't find the container with id 9c2d86935374fe01a6356fb5a01ccd59c0a4d0a6823af567d216b647d50aa793
Nov 03 22:48:09.377601 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.377569    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:48:09.377675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.377635    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:48:09.377675 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.377669    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:48:09.379437 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.379416    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"]
Nov 03 22:48:09.415281 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.415255    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" event={"ID":"a2911c96-53cb-4128-8f79-0975e1b97894","Type":"ContainerStarted","Data":"9c2d86935374fe01a6356fb5a01ccd59c0a4d0a6823af567d216b647d50aa793"}
Nov 03 22:48:09.509441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:09.508024    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:10.097928 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:48:10.097878    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:48:10.301916 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:10.301885    5162 patch_prober.go:29] interesting pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:48:10.302091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:10.301935    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:48:10.424090 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:10.424054    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" event={"ID":"a2911c96-53cb-4128-8f79-0975e1b97894","Type":"ContainerStarted","Data":"cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15"}
Nov 03 22:48:10.641427 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:10.640684    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podStartSLOduration=2.6406623380000003 podStartE2EDuration="2.640662338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:48:10.621464596 +0000 UTC m=+1493.886164608" watchObservedRunningTime="2026-11-03 22:48:10.640662338 +0000 UTC m=+1493.905362350"
Nov 03 22:48:10.917893 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:10.917868    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:10.919255 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:10.919234    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Nov 03 22:48:10.919416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:10.919382    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Nov 03 22:48:11.301940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:11.301850    5162 patch_prober.go:29] interesting pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:48:11.301940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:11.301918    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:48:11.326901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:11.326868    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:11.918321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:11.918222    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Nov 03 22:48:11.918321 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:11.918280    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Nov 03 22:48:12.294267 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:12.294174    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:12.302426 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:12.302378    5162 patch_prober.go:29] interesting pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:48:12.302885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:12.302454    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:48:12.405313 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:12.405284    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:48:12.495909 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:12.495871    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:12.919901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:12.919276    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Nov 03 22:48:12.919901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:12.919335    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Nov 03 22:48:13.305776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:13.305698    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:48:13.306268 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:13.306095    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:48:13.306780 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:13.306761    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:48:13.933724 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:13.933666    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Nov 03 22:48:13.934058 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:13.934017    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Nov 03 22:48:14.857414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:14.840771    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:14.919442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:14.918316    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Nov 03 22:48:14.919442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:14.918371    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Nov 03 22:48:15.918854 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:15.918818    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Nov 03 22:48:15.919333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:15.918878    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Nov 03 22:48:16.918839 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:16.918805    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Nov 03 22:48:16.919005 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:16.918877    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Nov 03 22:48:18.156291 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:18.156034    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:18.156808 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:18.156672    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:18.157487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:18.157469    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 22:48:20.114118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:20.114074    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:25.402238 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:25.402184    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:28.665865 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:28.665830    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"]
Nov 03 22:48:29.142601 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:29.132626    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:29.393467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:29.391104    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:30.042986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.042951    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:30.200309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.200280    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"]
Nov 03 22:48:30.204615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.204598    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.309564 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.309476    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-a03fee-serving-cert\""
Nov 03 22:48:30.331569 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.331537    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"]
Nov 03 22:48:30.339957 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.339899    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.340335 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.340305    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.340519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.340500    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/063a642b-b204-45bb-893b-0e685d5dabc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.340796 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.340779    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4c9j\" (UniqueName: \"kubernetes.io/projected/063a642b-b204-45bb-893b-0e685d5dabc8-kube-api-access-k4c9j\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.442096 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.442057    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.442296 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.442138    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/063a642b-b204-45bb-893b-0e685d5dabc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.442362 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.442319    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4c9j\" (UniqueName: \"kubernetes.io/projected/063a642b-b204-45bb-893b-0e685d5dabc8-kube-api-access-k4c9j\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.442516 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.442487    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.443123 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:48:30.443089    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-serving-cert: secret "maas-api-e2e-shared-b-w4-a03fee-serving-cert" not found
Nov 03 22:48:30.445507 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:48:30.443552    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls podName:063a642b-b204-45bb-893b-0e685d5dabc8 nodeName:}" failed. No retries permitted until 2026-11-03 22:48:30.943511405 +0000 UTC m=+1514.208211414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls") pod "maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww" (UID: "063a642b-b204-45bb-893b-0e685d5dabc8") : secret "maas-api-e2e-shared-b-w4-a03fee-serving-cert" not found
Nov 03 22:48:30.445507 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.444308    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/063a642b-b204-45bb-893b-0e685d5dabc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.447169 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.447144    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.507220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.507188    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4c9j\" (UniqueName: \"kubernetes.io/projected/063a642b-b204-45bb-893b-0e685d5dabc8-kube-api-access-k4c9j\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.572720 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.572625    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:30.808516 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.808488    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:30.975035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.974994    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:30.986176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:30.986062    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:31.031093 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:31.031040    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:31.118277 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:31.118234    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:31.510304 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:48:31.510271    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063a642b_b204_45bb_893b_0e685d5dabc8.slice/crio-da5234af0c2fc87c328a6dff121ede3a20185be9b15f935d964d83748a4a6ec9 WatchSource:0}: Error finding container da5234af0c2fc87c328a6dff121ede3a20185be9b15f935d964d83748a4a6ec9: Status 404 returned error can't find the container with id da5234af0c2fc87c328a6dff121ede3a20185be9b15f935d964d83748a4a6ec9
Nov 03 22:48:31.514573 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:31.514524    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"]
Nov 03 22:48:31.515918 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:31.515844    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:31.636180 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:31.636150    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww" event={"ID":"063a642b-b204-45bb-893b-0e685d5dabc8","Type":"ContainerStarted","Data":"da5234af0c2fc87c328a6dff121ede3a20185be9b15f935d964d83748a4a6ec9"}
Nov 03 22:48:32.648718 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:32.648677    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww" event={"ID":"063a642b-b204-45bb-893b-0e685d5dabc8","Type":"ContainerStarted","Data":"beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed"}
Nov 03 22:48:32.649208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:32.648830    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:32.790599 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:32.790501    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww" podStartSLOduration=1.8743931 podStartE2EDuration="2.790488152s" totalImagesPullingTime="916.095052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:48:32.788600563 +0000 UTC m=+1516.053300585" watchObservedRunningTime="2026-11-03 22:48:32.790488152 +0000 UTC m=+1516.055188165"
Nov 03 22:48:33.666712 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:33.666672    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x" podUID="43d89bb0-a280-440b-8def-3a640b5e2320" containerName="payload-processing" containerID="cri-o://dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5" gracePeriod=25
Nov 03 22:48:33.937833 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:33.937812    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:48:34.025979 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.025952    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43d89bb0-a280-440b-8def-3a640b5e2320-kube-api-access-l4cdx\" (UniqueName: \"kubernetes.io/projected/43d89bb0-a280-440b-8def-3a640b5e2320-kube-api-access-l4cdx\") pod \"43d89bb0-a280-440b-8def-3a640b5e2320\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") "
Nov 03 22:48:34.026117 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.025990    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43d89bb0-a280-440b-8def-3a640b5e2320-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d89bb0-a280-440b-8def-3a640b5e2320-plugins-config-volume\") pod \"43d89bb0-a280-440b-8def-3a640b5e2320\" (UID: \"43d89bb0-a280-440b-8def-3a640b5e2320\") "
Nov 03 22:48:34.026466 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.026437    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d89bb0-a280-440b-8def-3a640b5e2320-plugins-config-volume" pod "43d89bb0-a280-440b-8def-3a640b5e2320" (UID: "43d89bb0-a280-440b-8def-3a640b5e2320"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:48:34.026743 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.026727    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d89bb0-a280-440b-8def-3a640b5e2320-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:48:34.028018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.027998    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d89bb0-a280-440b-8def-3a640b5e2320-kube-api-access-l4cdx" pod "43d89bb0-a280-440b-8def-3a640b5e2320" (UID: "43d89bb0-a280-440b-8def-3a640b5e2320"). InnerVolumeSpecName "kube-api-access-l4cdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:48:34.128140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.128107    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4cdx\" (UniqueName: \"kubernetes.io/projected/43d89bb0-a280-440b-8def-3a640b5e2320-kube-api-access-l4cdx\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:48:34.668039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.668006    5162 generic.go:356] "Generic (PLEG): container finished" podID="43d89bb0-a280-440b-8def-3a640b5e2320" containerID="dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5" exitCode=0
Nov 03 22:48:34.668039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.668049    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"
Nov 03 22:48:34.668628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.668086    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x" event={"ID":"43d89bb0-a280-440b-8def-3a640b5e2320","Type":"ContainerDied","Data":"dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5"}
Nov 03 22:48:34.668628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.668146    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x" event={"ID":"43d89bb0-a280-440b-8def-3a640b5e2320","Type":"ContainerDied","Data":"e9a22fdd2c38b6d32d0fab7913f484fe7710a192d545d9197c2804884deb4cff"}
Nov 03 22:48:34.668628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.668181    5162 scope.go:122] "RemoveContainer" containerID="dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5"
Nov 03 22:48:34.679902 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.679886    5162 scope.go:122] "RemoveContainer" containerID="dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5"
Nov 03 22:48:34.680191 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:48:34.680173    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5\": container with ID starting with dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5 not found: ID does not exist" containerID="dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5"
Nov 03 22:48:34.680244 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.680199    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5"} err="failed to get container status \"dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5\": rpc error: code = NotFound desc = could not find container \"dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5\": container with ID starting with dfc0bf2286647117c23fced97542be9f27571b5b2ba52dac5dda03025d188ff5 not found: ID does not exist"
Nov 03 22:48:34.842140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.842059    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"]
Nov 03 22:48:34.846990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.846963    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x"]
Nov 03 22:48:34.866301 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:34.866270    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43d89bb0-a280-440b-8def-3a640b5e2320" path="/var/lib/kubelet/pods/43d89bb0-a280-440b-8def-3a640b5e2320/volumes"
Nov 03 22:48:35.715038 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:35.715002    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:38.222432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:38.219943    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:38.665076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:38.664997    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 22:48:40.085900 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:48:40.085856    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:48:40.516717 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:40.516681    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:45.356018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:45.355983    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:50.138179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:50.138140    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:51.472828 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:51.472786    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:54.946605 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:54.946565    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:48:59.703203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:48:59.703147    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:01.300342 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:01.300308    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:03.459875 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:03.459844    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:04.144897 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:04.144862    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:04.397304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:04.397211    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:04.744285 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:04.744250    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:08.727087 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:08.726990    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:08.938000 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:08.937969    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:09.921352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:09.921299    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:10.086220 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:49:10.086184    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:49:11.664938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:11.664901    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:12.010788 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:12.010758    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:12.221811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:12.221779    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:14.372299 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:14.372266    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:16.108713 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:16.108677    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:16.319245 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:16.319203    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:16.527538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:16.527507    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:16.759935 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:16.759900    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:17.571069 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:17.571035    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:18.091260 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:18.091227    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:18.342109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:18.342023    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:18.820349 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:18.820283    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:20.724958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:20.724925    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:22.018027 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:22.017994    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:22.679169 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:22.679133    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:23.360134 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:23.360093    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:23.577856 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:23.577818    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:27.153432 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:27.153376    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:27.431641 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:27.431539    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:27.655786 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:27.655747    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:29.101125 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:29.101062    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:29.303442 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:29.303378    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:32.205176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:32.205137    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:37.041385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:37.041344    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:40.085478 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:49:40.085442    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:49:40.646570 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:40.646535    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:41.881132 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:41.881096    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:42.141664 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:42.141560    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:42.676458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:42.676229    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:43.065452 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:43.065351    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:46.939098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:46.939044    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:47.162057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:47.162001    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:47.434938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:47.434904    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:48.915346 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:48.915310    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:51.321409 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:51.321350    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:51.520289 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:51.520254    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:51.850634 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:51.850598    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:53.006528 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:53.006425    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:55.320082 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:55.320044    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:49:58.281544 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:49:58.281496    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:03.413343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:03.413299    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:08.534247 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:08.534194    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:10.084232 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:50:10.084196    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:50:11.609710 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:11.609681    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:13.282014 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:13.281978    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:13.546289 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:13.546200    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:13.753097 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:13.753057    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:15.121821 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:15.121788    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:15.323652 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:15.323614    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:18.274565 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:18.274501    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:23.055221 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:23.055186    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:27.809228 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:27.809191    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:28.067013 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:28.066933    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:32.203844 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:32.203807    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:33.831745 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:33.831711    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:34.170210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:34.170182    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:34.376547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:34.376517    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:34.821400 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:34.821360    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:36.957685 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:36.957594    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:39.310797 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:39.310760    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:40.084809 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:50:40.084775    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:50:40.902976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:40.902940    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:41.105850 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:41.105801    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:41.180759 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:41.180731    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"]
Nov 03 22:50:41.180928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:41.180907    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" containerID="cri-o://aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563" gracePeriod=30
Nov 03 22:50:41.349532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:41.349369    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:42.484877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:42.484844    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:43.308798 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:43.308759    5162 patch_prober.go:29] interesting pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:50:43.308958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:43.308834    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:50:43.661143 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:43.660943    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:43.879844 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:43.879816    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:44.274243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:44.274206    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:46.479005 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.478981    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:50:46.597754 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.597726    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-envoy\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.597924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.597783    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-credential-socket\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.597924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.597827    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da0a843f-e5e3-43b7-93fd-f66a407d066e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da0a843f-e5e3-43b7-93fd-f66a407d066e-istiod-ca-cert\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.597924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.597881    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-kube-api-access-zdgxf\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-kube-api-access-zdgxf\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.598151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598058    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-certs\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.598151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598072    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-credential-socket" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:46.598151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598100    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-data\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.598151 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598135    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-socket\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.598361 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598208    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-podinfo\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.598361 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598245    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-token\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-token\") pod \"da0a843f-e5e3-43b7-93fd-f66a407d066e\" (UID: \"da0a843f-e5e3-43b7-93fd-f66a407d066e\") "
Nov 03 22:50:46.598361 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598276    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-certs" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:46.598361 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598205    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da0a843f-e5e3-43b7-93fd-f66a407d066e-istiod-ca-cert" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:50:46.598567 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598526    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-data" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:46.598567 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598544    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-socket" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:46.599010 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.598983    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.599116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.599014    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da0a843f-e5e3-43b7-93fd-f66a407d066e-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.599116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.599029    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.599116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.599042    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.599116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.599055    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.600049 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.600027    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-envoy" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:46.600310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.600290    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-kube-api-access-zdgxf" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "kube-api-access-zdgxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:46.600310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.600303    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-podinfo" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:50:46.600580 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.600566    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-token" pod "da0a843f-e5e3-43b7-93fd-f66a407d066e" (UID: "da0a843f-e5e3-43b7-93fd-f66a407d066e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:46.700958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.700930    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.700958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.700958    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.701139 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.700970    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da0a843f-e5e3-43b7-93fd-f66a407d066e-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:46.701139 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:46.700983    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdgxf\" (UniqueName: \"kubernetes.io/projected/da0a843f-e5e3-43b7-93fd-f66a407d066e-kube-api-access-zdgxf\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:47.050719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.050684    5162 generic.go:356] "Generic (PLEG): container finished" podID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerID="aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563" exitCode=0
Nov 03 22:50:47.050901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.050737    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" event={"ID":"da0a843f-e5e3-43b7-93fd-f66a407d066e","Type":"ContainerDied","Data":"aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563"}
Nov 03 22:50:47.050901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.050780    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"
Nov 03 22:50:47.050901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.050809    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc" event={"ID":"da0a843f-e5e3-43b7-93fd-f66a407d066e","Type":"ContainerDied","Data":"112846e92d5d55a8be8007f2a0bb63a5b487440d1c26c872a53ed0c3d29e007d"}
Nov 03 22:50:47.050901 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.050843    5162 scope.go:122] "RemoveContainer" containerID="aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563"
Nov 03 22:50:47.061943 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.061915    5162 scope.go:122] "RemoveContainer" containerID="12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1"
Nov 03 22:50:47.070899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.070878    5162 scope.go:122] "RemoveContainer" containerID="aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563"
Nov 03 22:50:47.071151 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:50:47.071133    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563\": container with ID starting with aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563 not found: ID does not exist" containerID="aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563"
Nov 03 22:50:47.071203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.071158    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563"} err="failed to get container status \"aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563\": rpc error: code = NotFound desc = could not find container \"aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563\": container with ID starting with aaf560b126855105a206dd2eb2a5e8896a139da408d9e423a7e7833beb64a563 not found: ID does not exist"
Nov 03 22:50:47.071203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.071174    5162 scope.go:122] "RemoveContainer" containerID="12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1"
Nov 03 22:50:47.071441 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:50:47.071419    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1\": container with ID starting with 12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1 not found: ID does not exist" containerID="12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1"
Nov 03 22:50:47.071505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.071450    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1"} err="failed to get container status \"12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1\": rpc error: code = NotFound desc = could not find container \"12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1\": container with ID starting with 12fb28f2c5e9a6a5156a0a754b623fc0f2336f5be5343f6795778a710558fbc1 not found: ID does not exist"
Nov 03 22:50:47.223006 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.222973    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"]
Nov 03 22:50:47.273950 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.273873    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc"]
Nov 03 22:50:47.828717 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:47.828684    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:48.090861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:48.090783    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:48.855631 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:48.855596    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" path="/var/lib/kubelet/pods/da0a843f-e5e3-43b7-93fd-f66a407d066e/volumes"
Nov 03 22:50:49.304528 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:49.304490    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:51.603687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:51.603650    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:52.241418 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:52.241071    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:52.322804 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:52.322767    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"]
Nov 03 22:50:52.323286 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:52.323233    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" containerID="cri-o://73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7" gracePeriod=30
Nov 03 22:50:52.711676 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:52.711642    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:52.939611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:52.939573    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:53.157122 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:53.157029    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:54.681403 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:54.681351    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:54.887578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:54.887539    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:55.099015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:55.098937    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:56.918798 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:56.918771    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:56.960045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:56.960007    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:50:56.960203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:56.960084    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:50:57.135701 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.135671    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:50:57.588721 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.588697    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:50:57.667453 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667360    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/02cc5b72-9510-4de2-be26-c2b2142af282-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02cc5b72-9510-4de2-be26-c2b2142af282-istio-podinfo\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.667579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667495    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02cc5b72-9510-4de2-be26-c2b2142af282-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02cc5b72-9510-4de2-be26-c2b2142af282-istiod-ca-cert\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.667579 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667566    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-certs\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.667653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667609    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-istio-token\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-istio-token\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.667726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667711    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-data\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.667785 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667747    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-credential-socket\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.667848 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667827    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-certs" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:57.667929 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667912    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-socket\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.667986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667943    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-data" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:57.667986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667961    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02cc5b72-9510-4de2-be26-c2b2142af282-istiod-ca-cert" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:50:57.667986 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.667982    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-kube-api-access-spq2c\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-kube-api-access-spq2c\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.668110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668021    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-envoy\") pod \"02cc5b72-9510-4de2-be26-c2b2142af282\" (UID: \"02cc5b72-9510-4de2-be26-c2b2142af282\") "
Nov 03 22:50:57.668110 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668025    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-credential-socket" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:57.668188 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668151    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-socket" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:57.668740 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668711    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.668740 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668738    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02cc5b72-9510-4de2-be26-c2b2142af282-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.668913 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668752    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.668913 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668765    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.668913 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.668778    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.669723 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.669697    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02cc5b72-9510-4de2-be26-c2b2142af282-istio-podinfo" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:50:57.669876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.669849    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-istio-token" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:57.670039 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.670020    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-kube-api-access-spq2c" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "kube-api-access-spq2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:57.670102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.670079    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-envoy" pod "02cc5b72-9510-4de2-be26-c2b2142af282" (UID: "02cc5b72-9510-4de2-be26-c2b2142af282"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:57.779414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.779204    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spq2c\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-kube-api-access-spq2c\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.779414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.779245    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02cc5b72-9510-4de2-be26-c2b2142af282-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.779414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.779259    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02cc5b72-9510-4de2-be26-c2b2142af282-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:57.779414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:57.779278    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02cc5b72-9510-4de2-be26-c2b2142af282-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:50:58.141060 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.141025    5162 generic.go:356] "Generic (PLEG): container finished" podID="02cc5b72-9510-4de2-be26-c2b2142af282" containerID="73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7" exitCode=0
Nov 03 22:50:58.141495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.141090    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"
Nov 03 22:50:58.141495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.141108    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" event={"ID":"02cc5b72-9510-4de2-be26-c2b2142af282","Type":"ContainerDied","Data":"73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7"}
Nov 03 22:50:58.141495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.141162    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc" event={"ID":"02cc5b72-9510-4de2-be26-c2b2142af282","Type":"ContainerDied","Data":"4e01858b35113e6019e9a514cf757bf1fd6209cb555f743cd44a764c307859c1"}
Nov 03 22:50:58.141495 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.141184    5162 scope.go:122] "RemoveContainer" containerID="73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7"
Nov 03 22:50:58.152594 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.152575    5162 scope.go:122] "RemoveContainer" containerID="73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7"
Nov 03 22:50:58.152861 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:50:58.152842    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7\": container with ID starting with 73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7 not found: ID does not exist" containerID="73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7"
Nov 03 22:50:58.152913 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.152868    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7"} err="failed to get container status \"73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7\": rpc error: code = NotFound desc = could not find container \"73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7\": container with ID starting with 73de91960e0eac108bef4dbdc3ba99d556800564b5af2906af6a09b03b8488d7 not found: ID does not exist"
Nov 03 22:50:58.312878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.312845    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"]
Nov 03 22:50:58.364317 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.364289    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc"]
Nov 03 22:50:58.856777 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:50:58.856747    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" path="/var/lib/kubelet/pods/02cc5b72-9510-4de2-be26-c2b2142af282/volumes"
Nov 03 22:51:00.095274 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:00.095237    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:02.015198 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:02.015164    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:02.217757 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:02.217725    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:05.026722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:05.026687    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:09.825919 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:09.825881    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:10.088856 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:51:10.088766    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:51:11.975455 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:11.975418    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:14.734787 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:14.734751    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:17.023782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:17.023738    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:17.185623 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:17.185593    5162 scope.go:122] "RemoveContainer" containerID="9ef2295c00dc632224b4c5253a32deb0a907b07719ebc6db59aeaf55114db416"
Nov 03 22:51:19.413400 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:19.413350    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:19.616415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:19.616337    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:20.221420 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:20.204679    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:20.772220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:20.772187    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:24.642149 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:24.642100    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:25.160172 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:25.160138    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:25.412445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:25.412365    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:28.517117 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:28.517087    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:28.725789 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:28.725753    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:29.972051 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:29.972009    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:31.640137 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:31.640100    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:32.114795 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:32.114758    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:32.880040 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:32.880004    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:33.107519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:33.107486    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:35.366770 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:35.366733    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:37.343018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:37.342979    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:37.601810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:37.601730    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:40.085098 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:51:40.085049    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:51:40.637360 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:40.637325    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:43.554831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:43.554802    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:43.836568 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:43.836440    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:45.729365 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:45.729307    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:46.616553 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:46.616519    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:47.092379 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:47.092340    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:47.773980 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:47.773945    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:48.297079 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:48.297048    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:50.946044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:50.945997    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:52.300772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:52.300738    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:52.559025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:52.558938    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:52.767185 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:52.767127    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:52.938325 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:52.938299    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"]
Nov 03 22:51:52.938530 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:52.938509    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" containerID="cri-o://48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d" gracePeriod=30
Nov 03 22:51:53.163306 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:53.163271    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"]
Nov 03 22:51:53.163541 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:53.163517    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy" containerID="cri-o://a48e573e2ce4ade093523efbec9a73c69b8f84c070aac8710538537fe8c22449" gracePeriod=30
Nov 03 22:51:53.786574 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:53.786543    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"]
Nov 03 22:51:53.786968 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:53.786747    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy" containerID="cri-o://2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3" gracePeriod=30
Nov 03 22:51:55.594355 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:55.594317    5162 patch_prober.go:29] interesting pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:51:55.594810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:55.594425    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:51:56.220884 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:56.220843    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:57.923946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:57.923910    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:58.135021 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.134991    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:58.426914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.426889    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508033    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-envoy\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508134    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-podinfo\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508174    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-data\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508255    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-socket\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508357    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istiod-ca-cert\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508419    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-kube-api-access-77lsk\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-kube-api-access-77lsk\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508459    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-token\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-token\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508499    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-credential-socket\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.509778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.508598    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-certs\") pod \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\" (UID: \"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc\") "
Nov 03 22:51:58.516454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.511083    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-socket" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.516454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.511308    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-data" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.516454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.511537    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-credential-socket" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.517052 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.517020    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-certs" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.524021 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.519051    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istiod-ca-cert" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:58.636319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.632736    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.636319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.632769    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.636319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.632783    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.636319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.632798    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.636319 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.632811    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.638416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.637656    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-kube-api-access-77lsk" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "kube-api-access-77lsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:58.638416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.638364    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-podinfo" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:58.659256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.657946    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-token" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:58.669002 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.668962    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-envoy" pod "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" (UID: "69aa4032-5ee5-4b3e-95c2-1e39feb71bcc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.692348 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.692152    5162 generic.go:356] "Generic (PLEG): container finished" podID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerID="a48e573e2ce4ade093523efbec9a73c69b8f84c070aac8710538537fe8c22449" exitCode=0
Nov 03 22:51:58.692348 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.692325    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" event={"ID":"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b","Type":"ContainerDied","Data":"a48e573e2ce4ade093523efbec9a73c69b8f84c070aac8710538537fe8c22449"}
Nov 03 22:51:58.696831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.695238    5162 generic.go:356] "Generic (PLEG): container finished" podID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerID="48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d" exitCode=0
Nov 03 22:51:58.696831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.695440    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" event={"ID":"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc","Type":"ContainerDied","Data":"48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d"}
Nov 03 22:51:58.696831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.695496    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm" event={"ID":"69aa4032-5ee5-4b3e-95c2-1e39feb71bcc","Type":"ContainerDied","Data":"b1db083719adf6758ccbbe1cbdcf29e5242714092b7f3d11bda89c1a96d10de8"}
Nov 03 22:51:58.696831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.695614    5162 scope.go:122] "RemoveContainer" containerID="48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d"
Nov 03 22:51:58.696831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.695793    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"
Nov 03 22:51:58.715777 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.715747    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:51:58.728285 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.728262    5162 scope.go:122] "RemoveContainer" containerID="5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09"
Nov 03 22:51:58.734255 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.734235    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-data\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.734577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.734561    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-token\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-token\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.734744 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.734728    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-certs\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.735199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.734848    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-data" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.735363 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.735345    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-socket\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.735593 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.735548    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-certs" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.735735 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.735658    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-socket" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.735835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.735699    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-podinfo\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.735930 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.735918    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-credential-socket\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.736190 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.736178    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-envoy\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.736472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.736460    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istiod-ca-cert\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.736570 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.736559    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-kube-api-access-btvjl\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-kube-api-access-btvjl\") pod \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\" (UID: \"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b\") "
Nov 03 22:51:58.737098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737085    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.737250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737240    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77lsk\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-kube-api-access-77lsk\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.737325 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737317    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.737380 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737372    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.737533 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737519    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.737619 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737608    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.738030 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.738015    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.738140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.736275    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-credential-socket" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.738246 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737743    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-podinfo" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:58.738326 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.737981    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istiod-ca-cert" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:58.738771 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.738754    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-token" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:58.740040 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.740021    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-envoy" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:58.741470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.741441    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-kube-api-access-btvjl" pod "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" (UID: "3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b"). InnerVolumeSpecName "kube-api-access-btvjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:58.758033 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.758013    5162 scope.go:122] "RemoveContainer" containerID="48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d"
Nov 03 22:51:58.758477 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:51:58.758456    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d\": container with ID starting with 48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d not found: ID does not exist" containerID="48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d"
Nov 03 22:51:58.758577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.758492    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d"} err="failed to get container status \"48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d\": rpc error: code = NotFound desc = could not find container \"48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d\": container with ID starting with 48823a7b4e8d9e6cc17f7a82449e5f3c15e76c04ab2c446c981799416b08735d not found: ID does not exist"
Nov 03 22:51:58.758577 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.758516    5162 scope.go:122] "RemoveContainer" containerID="5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09"
Nov 03 22:51:58.758880 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:51:58.758855    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09\": container with ID starting with 5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09 not found: ID does not exist" containerID="5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09"
Nov 03 22:51:58.758930 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.758892    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09"} err="failed to get container status \"5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09\": rpc error: code = NotFound desc = could not find container \"5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09\": container with ID starting with 5c36e3c0c1b24e8c7bf469df1eb83b3bf206b6de8ddf29643f316fab5323da09 not found: ID does not exist"
Nov 03 22:51:58.839262 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.839234    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.839262 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.839264    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.839412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.839274    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.839412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.839282    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.839412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.839291    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btvjl\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-kube-api-access-btvjl\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.839412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.839300    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:58.871984 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.871957    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"]
Nov 03 22:51:58.876654 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:58.876632    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm"]
Nov 03 22:51:59.083040 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.083002    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:51:59.142655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.142629    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-token\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-token\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.142826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.142662    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-credential-socket\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.142826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.142732    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-podinfo\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.142826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.142773    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-certs\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.142826 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.142811    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-envoy\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.143042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.142855    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-data\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.143042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.142880    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-socket\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.143042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143001    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fb20c55-3376-49c7-a038-2edaa64ec005-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb20c55-3376-49c7-a038-2edaa64ec005-istiod-ca-cert\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.143191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143044    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-credential-socket" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:59.143191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143083    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-kube-api-access-gk6mc\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-kube-api-access-gk6mc\") pod \"3fb20c55-3376-49c7-a038-2edaa64ec005\" (UID: \"3fb20c55-3376-49c7-a038-2edaa64ec005\") "
Nov 03 22:51:59.143296 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143271    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-socket" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:59.143421 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143367    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-data" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:59.143506 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143438    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fb20c55-3376-49c7-a038-2edaa64ec005-istiod-ca-cert" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:59.143598 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143500    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-certs" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:59.143813 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143792    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb20c55-3376-49c7-a038-2edaa64ec005-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.143878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143820    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.143878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143835    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.143878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143849    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.143878 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.143863    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.145041 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.145008    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-envoy" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:59.145279 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.145256    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-token" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:59.145373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.145316    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-kube-api-access-gk6mc" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "kube-api-access-gk6mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:59.145522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.145503    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-podinfo" pod "3fb20c55-3376-49c7-a038-2edaa64ec005" (UID: "3fb20c55-3376-49c7-a038-2edaa64ec005"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:59.245250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.245205    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk6mc\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-kube-api-access-gk6mc\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.245250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.245237    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.245250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.245246    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.245250 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.245254    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb20c55-3376-49c7-a038-2edaa64ec005-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:51:59.289324 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.289288    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:51:59.703070 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.703035    5162 generic.go:356] "Generic (PLEG): container finished" podID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerID="2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3" exitCode=0
Nov 03 22:51:59.703236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.703103    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"
Nov 03 22:51:59.703236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.703123    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" event={"ID":"3fb20c55-3376-49c7-a038-2edaa64ec005","Type":"ContainerDied","Data":"2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3"}
Nov 03 22:51:59.703236 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.703227    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74" event={"ID":"3fb20c55-3376-49c7-a038-2edaa64ec005","Type":"ContainerDied","Data":"6e0b73e984dc3f67cda73e3da82e39c7496bd5929416bcaec6f863acce8ae019"}
Nov 03 22:51:59.703441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.703263    5162 scope.go:122] "RemoveContainer" containerID="2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3"
Nov 03 22:51:59.704682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.704609    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"
Nov 03 22:51:59.704682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.704628    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc" event={"ID":"3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b","Type":"ContainerDied","Data":"fd1ce5d1f7fad4d88f8ed667af797d1b3c22522f65bd34b3276be77ffbf4d492"}
Nov 03 22:51:59.714203 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.714189    5162 scope.go:122] "RemoveContainer" containerID="2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3"
Nov 03 22:51:59.714458 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:51:59.714438    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3\": container with ID starting with 2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3 not found: ID does not exist" containerID="2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3"
Nov 03 22:51:59.714532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.714470    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3"} err="failed to get container status \"2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3\": rpc error: code = NotFound desc = could not find container \"2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3\": container with ID starting with 2ba66ebefcccfe56ff0ae60bcb430a8fa293718e312d13916214c19cc91db6e3 not found: ID does not exist"
Nov 03 22:51:59.714532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.714493    5162 scope.go:122] "RemoveContainer" containerID="a48e573e2ce4ade093523efbec9a73c69b8f84c070aac8710538537fe8c22449"
Nov 03 22:51:59.874651 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.874621    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"]
Nov 03 22:51:59.926210 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:51:59.926182    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc"]
Nov 03 22:52:00.042849 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:00.042741    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"]
Nov 03 22:52:00.046243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:00.046206    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74"]
Nov 03 22:52:00.855778 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:00.855749    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" path="/var/lib/kubelet/pods/3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b/volumes"
Nov 03 22:52:00.856244 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:00.856229    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" path="/var/lib/kubelet/pods/3fb20c55-3376-49c7-a038-2edaa64ec005/volumes"
Nov 03 22:52:00.856737 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:00.856721    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" path="/var/lib/kubelet/pods/69aa4032-5ee5-4b3e-95c2-1e39feb71bcc/volumes"
Nov 03 22:52:01.295100 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:01.295052    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:01.544043 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:01.544008    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:01.745015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:01.744978    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:03.823941 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:03.823912    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:04.080978 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:04.080897    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:06.583076 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:06.583016    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:10.088359 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:52:10.088279    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:52:11.778402 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:11.778343    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:14.186879 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:14.186841    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:16.892253 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:16.892206    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:17.101559 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:17.101516    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:17.226199 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:17.226171    5162 scope.go:122] "RemoveContainer" containerID="f2cebec6358f60e37ca7fb1ce4ee06dd9659dfa4dbf03b8bb2dba0bb41f71c42"
Nov 03 22:52:17.391185 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:17.391114    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:20.432985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:20.432936    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:21.364945 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.364912    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:21.832256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832221    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"]
Nov 03 22:52:21.832827 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832812    5162 cpu_manager.go:414] "containerMap: removing container" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy"
Nov 03 22:52:21.832870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832828    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy"
Nov 03 22:52:21.832870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832839    5162 cpu_manager.go:414] "containerMap: removing container" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy"
Nov 03 22:52:21.832870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832844    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy"
Nov 03 22:52:21.832870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832859    5162 cpu_manager.go:414] "containerMap: removing container" podUID="43d89bb0-a280-440b-8def-3a640b5e2320" containerName="payload-processing"
Nov 03 22:52:21.832870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832865    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43d89bb0-a280-440b-8def-3a640b5e2320" containerName="payload-processing"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832874    5162 cpu_manager.go:414] "containerMap: removing container" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832879    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832894    5162 cpu_manager.go:414] "containerMap: removing container" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832898    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832912    5162 cpu_manager.go:414] "containerMap: removing container" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832917    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832923    5162 cpu_manager.go:414] "containerMap: removing container" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy"
Nov 03 22:52:21.833018 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.832928    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833025    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833035    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="02cc5b72-9510-4de2-be26-c2b2142af282" containerName="istio-proxy"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833039    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833048    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="43d89bb0-a280-440b-8def-3a640b5e2320" containerName="payload-processing"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833053    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a06e6ff-98ca-4eee-bbfd-25ddfb195d8b" containerName="istio-proxy"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833060    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fb20c55-3376-49c7-a038-2edaa64ec005" containerName="istio-proxy"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833066    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy"
Nov 03 22:52:21.833243 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.833071    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="69aa4032-5ee5-4b3e-95c2-1e39feb71bcc" containerName="istio-proxy"
Nov 03 22:52:21.837832 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.837816    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:21.896070 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.895937    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2769cb-openshift-default-dockercfg-l5kwc\""
Nov 03 22:52:21.911540 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.911503    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-certs\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:21.911872 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.911849    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bd9n\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-kube-api-access-8bd9n\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:21.912102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.912084    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-podinfo\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:21.939415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:21.912226    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-data\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.000836 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.000799    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istiod-ca-cert\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.002691 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.002663    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-envoy\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.003881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.003851    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-token\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.004061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.004045    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-socket\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.004477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.004446    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-credential-socket\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.021408 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.013003    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:22.049606 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.048825    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"]
Nov 03 22:52:22.105639 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105537    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-certs\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.105639 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105582    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bd9n\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-kube-api-access-8bd9n\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.105639 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105623    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-podinfo\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.105940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105655    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-data\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.105940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105703    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istiod-ca-cert\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.105940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105738    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-envoy\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.105940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105809    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-token\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.105940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105829    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-socket\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.106213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.105964    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-certs\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.106213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.106040    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-credential-socket\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.106213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.106043    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-data\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.106376 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.106284    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-socket\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.106376 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.106298    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-credential-socket\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.106545 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.106480    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istiod-ca-cert\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.108376 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.108352    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-envoy\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.108730 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.108709    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-podinfo\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.178458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.177607    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bd9n\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-kube-api-access-8bd9n\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.178458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.178104    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-token\") pod \"e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") " pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.429054 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.429003    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:22.453998 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.453935    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:22.645119 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.645008    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:22.894648 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:52:22.894620    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6d9588_1dfd_4247_ad1a_9cdd6f7d7051.slice/crio-63eebde652dc0ffd33b16c8df4b2d72f489e11e1217056ca98ad4f5edbb21845 WatchSource:0}: Error finding container 63eebde652dc0ffd33b16c8df4b2d72f489e11e1217056ca98ad4f5edbb21845: Status 404 returned error can't find the container with id 63eebde652dc0ffd33b16c8df4b2d72f489e11e1217056ca98ad4f5edbb21845
Nov 03 22:52:22.897229 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.897192    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:52:22.897327 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.897262    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:52:22.897327 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.897291    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:52:22.898694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:22.898675    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"]
Nov 03 22:52:23.063875 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:23.063841    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" event={"ID":"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051","Type":"ContainerStarted","Data":"1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908"}
Nov 03 22:52:23.064053 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:23.063983    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" event={"ID":"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051","Type":"ContainerStarted","Data":"63eebde652dc0ffd33b16c8df4b2d72f489e11e1217056ca98ad4f5edbb21845"}
Nov 03 22:52:23.190689 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:23.190635    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podStartSLOduration=2.190622051 podStartE2EDuration="2.190622051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:52:23.187946067 +0000 UTC m=+1746.452646078" watchObservedRunningTime="2026-11-03 22:52:23.190622051 +0000 UTC m=+1746.455322060"
Nov 03 22:52:23.363381 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:23.363340    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:23.455237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:23.454841    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:23.462560 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:23.462530    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:23.592081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:23.592045    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:24.071348 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:24.071277    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:24.072275 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:24.072255    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:52:26.345377 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:26.345341    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:26.582280 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:26.582246    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:28.251644 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:28.251609    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:33.750090 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:33.749760    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:36.593575 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:36.593539    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:36.905468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:36.901573    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:38.794680 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:38.794644    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:40.085025 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:52:40.084991    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:52:43.640429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:43.640372    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:48.389015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:48.388978    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:48.655115 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:48.655036    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:48.855041 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:48.855009    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:52.917318 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:52.917279    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:52.953189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:52.953161    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"]
Nov 03 22:52:52.953373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:52.953340    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy" containerID="cri-o://89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0" gracePeriod=30
Nov 03 22:52:53.454234 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:53.454194    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:54.033328 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:54.033296    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"]
Nov 03 22:52:54.033853 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:54.033553    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" containerID="cri-o://bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651" gracePeriod=30
Nov 03 22:52:57.849454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:57.849419    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:52:57.849942 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:57.849479    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:52:58.234688 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.234665    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:52:58.317700 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.317666    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-credential-socket\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.317881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.317823    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-envoy\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.317881 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.317858    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istiod-ca-cert\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.317985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.317885    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-data\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.317985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.317915    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-podinfo\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.317985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.317948    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-credential-socket" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:58.318138 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318036    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-socket\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.318138 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318098    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-certs\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.318138 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318125    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-token\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-token\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.318300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318217    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-data" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:58.318300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318225    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-kube-api-access-96td9\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-kube-api-access-96td9\") pod \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\" (UID: \"a23c229a-024d-422e-9c71-2b2c4abdfbbd\") "
Nov 03 22:52:58.318300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318265    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-socket" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:58.318888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318296    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istiod-ca-cert" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:52:58.318888 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.318417    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-certs" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:58.319025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.319001    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.319089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.319031    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.319089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.319045    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.319089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.319059    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.319089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.319072    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.320333 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.320299    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-podinfo" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:52:58.320641 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.320609    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-kube-api-access-96td9" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "kube-api-access-96td9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:52:58.320641 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.320622    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-envoy" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:58.320996 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.320975    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-token" pod "a23c229a-024d-422e-9c71-2b2c4abdfbbd" (UID: "a23c229a-024d-422e-9c71-2b2c4abdfbbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:52:58.420307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.420274    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.420307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.420301    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.420307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.420310    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.420624 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.420320    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96td9\" (UniqueName: \"kubernetes.io/projected/a23c229a-024d-422e-9c71-2b2c4abdfbbd-kube-api-access-96td9\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:58.428056 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.428029    5162 generic.go:356] "Generic (PLEG): container finished" podID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerID="89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0" exitCode=0
Nov 03 22:52:58.428191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.428081    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"
Nov 03 22:52:58.428191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.428120    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" event={"ID":"a23c229a-024d-422e-9c71-2b2c4abdfbbd","Type":"ContainerDied","Data":"89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0"}
Nov 03 22:52:58.428191 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.428179    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr" event={"ID":"a23c229a-024d-422e-9c71-2b2c4abdfbbd","Type":"ContainerDied","Data":"654f22924b661d959f9fafd81cc69c8c5fdf7082204389894aa493376ff6db51"}
Nov 03 22:52:58.428356 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.428207    5162 scope.go:122] "RemoveContainer" containerID="89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0"
Nov 03 22:52:58.441781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.441695    5162 scope.go:122] "RemoveContainer" containerID="89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0"
Nov 03 22:52:58.442173 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:52:58.442140    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0\": container with ID starting with 89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0 not found: ID does not exist" containerID="89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0"
Nov 03 22:52:58.442536 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.442181    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0"} err="failed to get container status \"89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0\": rpc error: code = NotFound desc = could not find container \"89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0\": container with ID starting with 89c14bb6f9124756f3aeba46b15d111806d3af662a25665feb7ba94a683cc2f0 not found: ID does not exist"
Nov 03 22:52:58.504811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.504781    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:52:58.605590 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.605554    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"]
Nov 03 22:52:58.609246 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.609220    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr"]
Nov 03 22:52:58.855802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:58.855728    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" path="/var/lib/kubelet/pods/a23c229a-024d-422e-9c71-2b2c4abdfbbd/volumes"
Nov 03 22:52:59.316249 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.316228    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:52:59.335179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335142    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-credential-socket\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335274    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-socket\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335350 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335337    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-token\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-token\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335418    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-podinfo\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335433    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-credential-socket" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:59.335551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335465    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-envoy\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335551 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335531    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-kube-api-access-6dgmx\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-kube-api-access-6dgmx\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335572    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-socket" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:59.335628 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335601    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/037e749e-a34d-4610-a3da-9eaef32a46ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037e749e-a34d-4610-a3da-9eaef32a46ab-istiod-ca-cert\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335726 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335642    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-data\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.335790 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.335757    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-certs\") pod \"037e749e-a34d-4610-a3da-9eaef32a46ab\" (UID: \"037e749e-a34d-4610-a3da-9eaef32a46ab\") "
Nov 03 22:52:59.336124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336050    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-data" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:59.336124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336073    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037e749e-a34d-4610-a3da-9eaef32a46ab-istiod-ca-cert" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:52:59.336124 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336098    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-certs" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:59.336507 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336487    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037e749e-a34d-4610-a3da-9eaef32a46ab-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.336567 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336513    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.336567 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336527    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.336567 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336544    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.336567 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.336557    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.337805 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.337779    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-token" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:52:59.337989 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.337972    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-podinfo" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:52:59.338083 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.338068    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-envoy" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:52:59.338473 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.338456    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-kube-api-access-6dgmx" pod "037e749e-a34d-4610-a3da-9eaef32a46ab" (UID: "037e749e-a34d-4610-a3da-9eaef32a46ab"). InnerVolumeSpecName "kube-api-access-6dgmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:52:59.436869 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.436836    5162 generic.go:356] "Generic (PLEG): container finished" podID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerID="bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651" exitCode=0
Nov 03 22:52:59.437142 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.436951    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"
Nov 03 22:52:59.437142 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.436972    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" event={"ID":"037e749e-a34d-4610-a3da-9eaef32a46ab","Type":"ContainerDied","Data":"bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651"}
Nov 03 22:52:59.437142 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.437065    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p" event={"ID":"037e749e-a34d-4610-a3da-9eaef32a46ab","Type":"ContainerDied","Data":"b8fa7a8dfb655857e962df9bbbb4e24d2f9e1d0cd24926c69869d5f3c57784c9"}
Nov 03 22:52:59.437142 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.437124    5162 scope.go:122] "RemoveContainer" containerID="bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651"
Nov 03 22:52:59.438346 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.438280    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.438346 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.438305    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.438346 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.438320    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037e749e-a34d-4610-a3da-9eaef32a46ab-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.438346 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.438335    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dgmx\" (UniqueName: \"kubernetes.io/projected/037e749e-a34d-4610-a3da-9eaef32a46ab-kube-api-access-6dgmx\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:52:59.450476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.450012    5162 scope.go:122] "RemoveContainer" containerID="bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651"
Nov 03 22:52:59.450476 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:52:59.450296    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651\": container with ID starting with bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651 not found: ID does not exist" containerID="bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651"
Nov 03 22:52:59.450476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.450332    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651"} err="failed to get container status \"bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651\": rpc error: code = NotFound desc = could not find container \"bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651\": container with ID starting with bc4344042cfed82833db513dd1c3c8ec80d45e3deb1562fa58fada557a433651 not found: ID does not exist"
Nov 03 22:52:59.621476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.621445    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"]
Nov 03 22:52:59.624913 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:52:59.624888    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p"]
Nov 03 22:53:00.855789 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:00.855749    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" path="/var/lib/kubelet/pods/037e749e-a34d-4610-a3da-9eaef32a46ab/volumes"
Nov 03 22:53:03.255967 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:03.255922    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:03.460937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:03.460897    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:03.662109 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:03.662022    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:08.241637 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:08.241523    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:10.085091 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:53:10.085051    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:53:12.744499 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:12.744463    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:13.092751 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:13.092651    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:17.731348 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:17.731307    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:18.021426 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:18.020820    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:22.738964 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:22.738875    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:22.956492 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:22.956454    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:23.164366 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:23.164271    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:28.037611 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:28.037572    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:32.923177 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:32.923137    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:33.747547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:33.747513    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:34.027074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:34.027000    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:37.743694 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:37.743608    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:40.086740 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:53:40.086698    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:53:41.868861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:41.868826    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:42.513467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:42.513428    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:43.141416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:43.095424    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:43.306114 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:43.305980    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:43.740804 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:43.740776    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:47.537231 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:47.537199    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:47.751620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:47.751580    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:48.365279 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:48.365246    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:52.730296 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:52.730252    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:52.985304 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:52.985224    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:53.829687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:53.829637    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:55.642176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:55.642119    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:56.108711 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:56.108677    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:57.118074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:57.118036    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:57.702424 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:57.702378    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:59.363415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:59.356845    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:59.583074 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:59.583037    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:53:59.935658 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:53:59.935621    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:01.760464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:01.760430    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:02.009909 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:02.009876    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:03.484372 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:03.484334    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:05.202976 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:05.202940    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:08.286487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.286439    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:08.577300 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.576082    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:08.657435 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.657359    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"]
Nov 03 22:54:08.658084 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658062    5162 cpu_manager.go:414] "containerMap: removing container" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy"
Nov 03 22:54:08.658208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658085    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy"
Nov 03 22:54:08.658208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658101    5162 cpu_manager.go:414] "containerMap: removing container" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy"
Nov 03 22:54:08.658208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658108    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da0a843f-e5e3-43b7-93fd-f66a407d066e" containerName="istio-proxy"
Nov 03 22:54:08.658208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658179    5162 cpu_manager.go:414] "containerMap: removing container" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy"
Nov 03 22:54:08.658208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658187    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy"
Nov 03 22:54:08.658505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658366    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="037e749e-a34d-4610-a3da-9eaef32a46ab" containerName="istio-proxy"
Nov 03 22:54:08.658505 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.658385    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="a23c229a-024d-422e-9c71-2b2c4abdfbbd" containerName="istio-proxy"
Nov 03 22:54:08.662077 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.662058    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.715777 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.715744    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"]
Nov 03 22:54:08.762046 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.761992    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzkql\" (UniqueName: \"kubernetes.io/projected/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-kube-api-access-tzkql\") pod \"payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.762256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.762133    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.764460 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.764430    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-9674f5\""
Nov 03 22:54:08.767604 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.767581    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-9674f5-dockercfg-rtpcw\""
Nov 03 22:54:08.803343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.798475    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"]
Nov 03 22:54:08.806078 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.806053    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:08.889467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.876157    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.889467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.876435    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzkql\" (UniqueName: \"kubernetes.io/projected/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-kube-api-access-tzkql\") pod \"payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.893269 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.893234    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.913413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.906285    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"]
Nov 03 22:54:08.965426 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.964475    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzkql\" (UniqueName: \"kubernetes.io/projected/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-kube-api-access-tzkql\") pod \"payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.980870 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.980637    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:08.981563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.981528    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a5f9047-afa4-4610-bcb3-866aac6cfe00-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:08.981716 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:08.981700    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57v5l\" (UniqueName: \"kubernetes.io/projected/3a5f9047-afa4-4610-bcb3-866aac6cfe00-kube-api-access-57v5l\") pod \"payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:09.084244 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:09.084204    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57v5l\" (UniqueName: \"kubernetes.io/projected/3a5f9047-afa4-4610-bcb3-866aac6cfe00-kube-api-access-57v5l\") pod \"payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:09.084548 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:09.084530    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a5f9047-afa4-4610-bcb3-866aac6cfe00-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:09.085596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:09.085567    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a5f9047-afa4-4610-bcb3-866aac6cfe00-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:09.147237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:09.147164    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57v5l\" (UniqueName: \"kubernetes.io/projected/3a5f9047-afa4-4610-bcb3-866aac6cfe00-kube-api-access-57v5l\") pod \"payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:09.366930 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:54:09.366889    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ba7a45_61ff_4f1a_aaba_709ee7c8839a.slice/crio-e1b2662b5473166057b0629477beb704ffd0b3a019e8dcfc658c5dbb45f58377 WatchSource:0}: Error finding container e1b2662b5473166057b0629477beb704ffd0b3a019e8dcfc658c5dbb45f58377: Status 404 returned error can't find the container with id e1b2662b5473166057b0629477beb704ffd0b3a019e8dcfc658c5dbb45f58377
Nov 03 22:54:09.371042 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:09.371013    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"]
Nov 03 22:54:09.442997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:09.442975    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:09.826828 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:54:09.826798    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5f9047_afa4_4610_bcb3_866aac6cfe00.slice/crio-d138584ce5b6b2f4d149df8ef5093137258cdefeb0c96f67f4a88057d00d9501 WatchSource:0}: Error finding container d138584ce5b6b2f4d149df8ef5093137258cdefeb0c96f67f4a88057d00d9501: Status 404 returned error can't find the container with id d138584ce5b6b2f4d149df8ef5093137258cdefeb0c96f67f4a88057d00d9501
Nov 03 22:54:09.830966 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:09.830941    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"]
Nov 03 22:54:10.071002 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.070892    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx" event={"ID":"27ba7a45-61ff-4f1a-aaba-709ee7c8839a","Type":"ContainerStarted","Data":"2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48"}
Nov 03 22:54:10.071002 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.070962    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx" event={"ID":"27ba7a45-61ff-4f1a-aaba-709ee7c8839a","Type":"ContainerStarted","Data":"e1b2662b5473166057b0629477beb704ffd0b3a019e8dcfc658c5dbb45f58377"}
Nov 03 22:54:10.071310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.071134    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:10.073235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.073169    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl" event={"ID":"3a5f9047-afa4-4610-bcb3-866aac6cfe00","Type":"ContainerStarted","Data":"2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13"}
Nov 03 22:54:10.073235 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.073211    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl" event={"ID":"3a5f9047-afa4-4610-bcb3-866aac6cfe00","Type":"ContainerStarted","Data":"d138584ce5b6b2f4d149df8ef5093137258cdefeb0c96f67f4a88057d00d9501"}
Nov 03 22:54:10.073430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.073265    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:10.086489 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:54:10.086460    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:54:10.193727 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.193677    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx" podStartSLOduration=2.193660823 podStartE2EDuration="2.193660823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:54:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:54:10.190544303 +0000 UTC m=+1853.455244315" watchObservedRunningTime="2026-11-03 22:54:10.193660823 +0000 UTC m=+1853.458360835"
Nov 03 22:54:10.325496 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.322571    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl" podStartSLOduration=2.322549677 podStartE2EDuration="2.322549677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:54:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:54:10.315444842 +0000 UTC m=+1853.580144855" watchObservedRunningTime="2026-11-03 22:54:10.322549677 +0000 UTC m=+1853.587249688"
Nov 03 22:54:10.771736 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:10.771693    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:13.082035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:13.081994    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:16.362925 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:16.362885    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:18.079858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:18.079819    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:21.083609 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:21.083578    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:54:21.084027 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:21.083982    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:54:21.599405 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:21.598800    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:26.926411 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:26.926295    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:31.547458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:31.547421    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:31.762011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:31.761975    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:31.964607 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:31.964558    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:34.300280 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:34.300252    5162 generic.go:356] "Generic (PLEG): container finished" podID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerID="1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908" exitCode=0
Nov 03 22:54:34.300687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:34.300323    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" event={"ID":"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051","Type":"ContainerDied","Data":"1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908"}
Nov 03 22:54:34.300772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:34.300757    5162 scope.go:122] "RemoveContainer" containerID="1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908"
Nov 03 22:54:34.302625 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:34.302597    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:54:34.302704 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:34.302669    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:54:34.302741 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:34.302703    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:54:35.310819 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:35.310787    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" event={"ID":"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051","Type":"ContainerStarted","Data":"56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3"}
Nov 03 22:54:35.461416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:35.456808    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:54:35.461613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:35.461551    5162 patch_prober.go:29] interesting pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:54:35.461680 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:35.461606    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:54:36.457548 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:36.457512    5162 patch_prober.go:29] interesting pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:54:36.458179 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:36.458138    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:54:36.954412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:36.953009    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:37.458413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:37.457765    5162 patch_prober.go:29] interesting pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:54:37.458413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:37.457825    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:54:37.809173 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:37.809088    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:38.460885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:38.456790    5162 patch_prober.go:29] interesting pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:54:38.460885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:38.456865    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:54:39.071953 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:39.071918    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:54:39.437650 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:39.437614    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"]
Nov 03 22:54:39.437951 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:39.437897    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy" containerID="cri-o://a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717" gracePeriod=30
Nov 03 22:54:39.465416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:39.461851    5162 patch_prober.go:29] interesting pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:54:39.465416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:39.461964    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:54:39.581118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:39.581079    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:54:39.581279 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:39.581136    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:54:40.096931 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:54:40.096888    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:54:40.465593 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:40.465548    5162 patch_prober.go:29] interesting pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:54:40.466068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:40.465631    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:54:41.369532 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.369502    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"]
Nov 03 22:54:41.369710 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.369684    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4" podUID="132faa2b-51ce-4007-b4f4-dee77876a808" containerName="maas-api" containerID="cri-o://a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992" gracePeriod=30
Nov 03 22:54:41.460423 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.460379    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:54:41.461582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.461558    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 22:54:41.554178 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.554028    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:41.667455 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.667426    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:54:41.772863 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.772833    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/132faa2b-51ce-4007-b4f4-dee77876a808-kube-api-access-p6vdf\" (UniqueName: \"kubernetes.io/projected/132faa2b-51ce-4007-b4f4-dee77876a808-kube-api-access-p6vdf\") pod \"132faa2b-51ce-4007-b4f4-dee77876a808\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") "
Nov 03 22:54:41.773063 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.772915    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-maas-api-tls\") pod \"132faa2b-51ce-4007-b4f4-dee77876a808\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") "
Nov 03 22:54:41.773063 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.772958    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/132faa2b-51ce-4007-b4f4-dee77876a808-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/132faa2b-51ce-4007-b4f4-dee77876a808-openshift-service-ca\") pod \"132faa2b-51ce-4007-b4f4-dee77876a808\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") "
Nov 03 22:54:41.773063 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.772984    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-metrics-certs\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-metrics-certs\") pod \"132faa2b-51ce-4007-b4f4-dee77876a808\" (UID: \"132faa2b-51ce-4007-b4f4-dee77876a808\") "
Nov 03 22:54:41.773377 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.773343    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132faa2b-51ce-4007-b4f4-dee77876a808-openshift-service-ca" pod "132faa2b-51ce-4007-b4f4-dee77876a808" (UID: "132faa2b-51ce-4007-b4f4-dee77876a808"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:41.773724 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.773704    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/132faa2b-51ce-4007-b4f4-dee77876a808-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:41.775253 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.775234    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/132faa2b-51ce-4007-b4f4-dee77876a808-kube-api-access-p6vdf" pod "132faa2b-51ce-4007-b4f4-dee77876a808" (UID: "132faa2b-51ce-4007-b4f4-dee77876a808"). InnerVolumeSpecName "kube-api-access-p6vdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:41.775571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.775542    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-metrics-certs" pod "132faa2b-51ce-4007-b4f4-dee77876a808" (UID: "132faa2b-51ce-4007-b4f4-dee77876a808"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:54:41.775658 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.775568    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-maas-api-tls" pod "132faa2b-51ce-4007-b4f4-dee77876a808" (UID: "132faa2b-51ce-4007-b4f4-dee77876a808"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:54:41.875323 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.875242    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6vdf\" (UniqueName: \"kubernetes.io/projected/132faa2b-51ce-4007-b4f4-dee77876a808-kube-api-access-p6vdf\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:41.875323 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.875273    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:41.875323 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:41.875283    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/132faa2b-51ce-4007-b4f4-dee77876a808-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:42.387571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.387532    5162 generic.go:356] "Generic (PLEG): container finished" podID="132faa2b-51ce-4007-b4f4-dee77876a808" containerID="a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992" exitCode=0
Nov 03 22:54:42.387782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.387581    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"
Nov 03 22:54:42.387782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.387597    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4" event={"ID":"132faa2b-51ce-4007-b4f4-dee77876a808","Type":"ContainerDied","Data":"a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992"}
Nov 03 22:54:42.387782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.387669    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4" event={"ID":"132faa2b-51ce-4007-b4f4-dee77876a808","Type":"ContainerDied","Data":"8fbd51b20fec4eeeca2a1d9f14501146eefffb9c91d2cfd21353d352104a09e0"}
Nov 03 22:54:42.387782 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.387699    5162 scope.go:122] "RemoveContainer" containerID="a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992"
Nov 03 22:54:42.411062 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.401730    5162 scope.go:122] "RemoveContainer" containerID="a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992"
Nov 03 22:54:42.411062 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:54:42.402088    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992\": container with ID starting with a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992 not found: ID does not exist" containerID="a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992"
Nov 03 22:54:42.411062 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.402120    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992"} err="failed to get container status \"a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992\": rpc error: code = NotFound desc = could not find container \"a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992\": container with ID starting with a40d6d1c6fc05eac1af15a3464e9e38ed336df04b35ed496c8005d69c5bc3992 not found: ID does not exist"
Nov 03 22:54:42.568864 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.568823    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"]
Nov 03 22:54:42.617590 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.617559    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12450a06-68776f84c-xr7x4"]
Nov 03 22:54:42.855784 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:42.855755    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="132faa2b-51ce-4007-b4f4-dee77876a808" path="/var/lib/kubelet/pods/132faa2b-51ce-4007-b4f4-dee77876a808/volumes"
Nov 03 22:54:43.163209 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:43.163130    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:43.461880 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:43.461845    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:43.673705 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:43.673669    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:43.970891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:43.958039    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:44.168301 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.168260    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:44.632997 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.632975    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:54:44.723781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.723712    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-envoy\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.723781 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.723761    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-certs\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.723806    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-credential-socket\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.723836    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-token\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-token\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.723873    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-data\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724075    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istiod-ca-cert\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724137    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-certs" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724156    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-data" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724190    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-credential-socket" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724244    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-kube-api-access-h54xp\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-kube-api-access-h54xp\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724295 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724290    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-socket\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724713 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724450    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-podinfo\") pod \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\" (UID: \"21fbf7f8-43dd-451b-973a-fdf8252cafa6\") "
Nov 03 22:54:44.724713 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724496    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istiod-ca-cert" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:44.724713 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.724593    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-socket" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:44.725082 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.725060    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.725082 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.725082    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.725237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.725092    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.725237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.725105    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.725237 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.725114    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.726474 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.726450    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-envoy" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:44.726562 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.726495    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-token" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:44.726610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.726568    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-kube-api-access-h54xp" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "kube-api-access-h54xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:44.726779 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.726762    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-podinfo" pod "21fbf7f8-43dd-451b-973a-fdf8252cafa6" (UID: "21fbf7f8-43dd-451b-973a-fdf8252cafa6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:54:44.826890 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.826855    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.826890 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.826894    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h54xp\" (UniqueName: \"kubernetes.io/projected/21fbf7f8-43dd-451b-973a-fdf8252cafa6-kube-api-access-h54xp\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.827087 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.826912    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:44.827087 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:44.826926    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21fbf7f8-43dd-451b-973a-fdf8252cafa6-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:45.414803 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.414764    5162 generic.go:356] "Generic (PLEG): container finished" podID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerID="a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717" exitCode=0
Nov 03 22:54:45.414803 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.414806    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"
Nov 03 22:54:45.415025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.414798    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" event={"ID":"21fbf7f8-43dd-451b-973a-fdf8252cafa6","Type":"ContainerDied","Data":"a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717"}
Nov 03 22:54:45.415025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.414925    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7" event={"ID":"21fbf7f8-43dd-451b-973a-fdf8252cafa6","Type":"ContainerDied","Data":"284a911414ea7d2e11758d9259fd397517d0f50496142d7347316d7169c6e8dd"}
Nov 03 22:54:45.415025 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.414956    5162 scope.go:122] "RemoveContainer" containerID="a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717"
Nov 03 22:54:45.438597 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.430728    5162 scope.go:122] "RemoveContainer" containerID="a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717"
Nov 03 22:54:45.438597 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:54:45.431911    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717\": container with ID starting with a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717 not found: ID does not exist" containerID="a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717"
Nov 03 22:54:45.438597 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.432032    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717"} err="failed to get container status \"a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717\": rpc error: code = NotFound desc = could not find container \"a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717\": container with ID starting with a3a8f5f2ac0b42e14b5fc2489073b48154e0fd879caa5e5425384957fe987717 not found: ID does not exist"
Nov 03 22:54:45.639028 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.638999    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"]
Nov 03 22:54:45.642436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:45.642416    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7"]
Nov 03 22:54:46.857938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:46.857906    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" path="/var/lib/kubelet/pods/21fbf7f8-43dd-451b-973a-fdf8252cafa6/volumes"
Nov 03 22:54:47.361749 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:47.361707    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"]
Nov 03 22:54:47.361938 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:47.361912    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" containerID="cri-o://a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931" gracePeriod=30
Nov 03 22:54:47.417053 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:47.417022    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"]
Nov 03 22:54:47.417287 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:47.417262    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" containerID="cri-o://617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f" gracePeriod=30
Nov 03 22:54:48.807810 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:48.807776    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:51.205414 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:51.205351    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:51.412722 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:51.412672    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:51.779905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:51.779870    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:54:51.780073 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:51.779925    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:54:51.797858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:51.797825    5162 patch_prober.go:29] interesting pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:54:51.797975 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:51.797885    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:54:52.779430 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.779403    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:54:52.821561 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.821537    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:54:52.855121 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855097    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-podinfo\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855151    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-credential-socket\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855176    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-kube-api-access-tszbr\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-kube-api-access-tszbr\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855273 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855231    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istiod-ca-cert\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855287    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-socket\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855335    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-envoy\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855483 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855439    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-data\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855518    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-certs\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855580    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-credential-socket\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855685 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855627    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-socket\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855685 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855661    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-token\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-token\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855711    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-data\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855741    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-certs\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855801    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-token\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-token\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855831    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-podinfo\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.855904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855515    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-credential-socket" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.855904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855859    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8826e5c8-7b53-4394-8f91-df31cab2db1d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8826e5c8-7b53-4394-8f91-df31cab2db1d-istiod-ca-cert\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.855904 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855901    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-envoy\") pod \"8826e5c8-7b53-4394-8f91-df31cab2db1d\" (UID: \"8826e5c8-7b53-4394-8f91-df31cab2db1d\") "
Nov 03 22:54:52.856168 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.856020    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-kube-api-access-9l7gr\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-kube-api-access-9l7gr\") pod \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\" (UID: \"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72\") "
Nov 03 22:54:52.856234 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855706    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istiod-ca-cert" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:52.856234 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855839    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-certs" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.856334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855885    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-credential-socket" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.856334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.855926    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-socket" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.856334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.856033    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-data" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.856334 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.856135    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-socket" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.856574 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.856331    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-certs" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.856685 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.856580    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8826e5c8-7b53-4394-8f91-df31cab2db1d-istiod-ca-cert" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:52.856979 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.856955    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.856986    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857001    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857016    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857031    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8826e5c8-7b53-4394-8f91-df31cab2db1d-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857046    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857068 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857063    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857380 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857076    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857380 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857092    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.857380 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857335    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-data" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.857963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.857927    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-podinfo" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:54:52.858941 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.858917    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-envoy" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.859511 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.859483    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-token" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:52.859835 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.859815    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-token" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:52.859946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.859845    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-envoy" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:52.859946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.859858    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-kube-api-access-tszbr" pod "8826e5c8-7b53-4394-8f91-df31cab2db1d" (UID: "8826e5c8-7b53-4394-8f91-df31cab2db1d"). InnerVolumeSpecName "kube-api-access-tszbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:52.859946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.859888    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-podinfo" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:54:52.860989 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.860971    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-kube-api-access-9l7gr" pod "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" (UID: "d7bd07ac-ea7b-4be4-8c87-35d96d7bce72"). InnerVolumeSpecName "kube-api-access-9l7gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:52.958099 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958057    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958099 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958090    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958099 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958104    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958401 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958120    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958401 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958134    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958401 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958156    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958401 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958172    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l7gr\" (UniqueName: \"kubernetes.io/projected/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72-kube-api-access-9l7gr\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958401 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958189    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8826e5c8-7b53-4394-8f91-df31cab2db1d-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:52.958401 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:52.958202    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tszbr\" (UniqueName: \"kubernetes.io/projected/8826e5c8-7b53-4394-8f91-df31cab2db1d-kube-api-access-tszbr\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:53.477472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.477433    5162 generic.go:356] "Generic (PLEG): container finished" podID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerID="617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f" exitCode=0
Nov 03 22:54:53.477657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.477503    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"
Nov 03 22:54:53.477657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.477515    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" event={"ID":"8826e5c8-7b53-4394-8f91-df31cab2db1d","Type":"ContainerDied","Data":"617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f"}
Nov 03 22:54:53.477657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.477613    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm" event={"ID":"8826e5c8-7b53-4394-8f91-df31cab2db1d","Type":"ContainerDied","Data":"ea0bb0db4b738594a56dc2df2ef9aed8a9ccf9674285ebcd03eb886c08946617"}
Nov 03 22:54:53.477657 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.477642    5162 scope.go:122] "RemoveContainer" containerID="617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f"
Nov 03 22:54:53.479155 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.479131    5162 generic.go:356] "Generic (PLEG): container finished" podID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerID="a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931" exitCode=0
Nov 03 22:54:53.479320 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.479304    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" event={"ID":"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72","Type":"ContainerDied","Data":"a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931"}
Nov 03 22:54:53.479443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.479347    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr" event={"ID":"d7bd07ac-ea7b-4be4-8c87-35d96d7bce72","Type":"ContainerDied","Data":"6c14fd4ad946ef45c0c734f497106224f92a1d33718167df186235366edc65f6"}
Nov 03 22:54:53.479521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.479487    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"
Nov 03 22:54:53.491614 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.491579    5162 scope.go:122] "RemoveContainer" containerID="617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f"
Nov 03 22:54:53.492246 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:54:53.492215    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f\": container with ID starting with 617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f not found: ID does not exist" containerID="617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f"
Nov 03 22:54:53.492328 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.492262    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f"} err="failed to get container status \"617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f\": rpc error: code = NotFound desc = could not find container \"617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f\": container with ID starting with 617c01e9f07ebcf7d04af803ed9591b9df5b33008062ce78ca4520a043bbeb9f not found: ID does not exist"
Nov 03 22:54:53.492328 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.492310    5162 scope.go:122] "RemoveContainer" containerID="a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931"
Nov 03 22:54:53.505057 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.505037    5162 scope.go:122] "RemoveContainer" containerID="a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931"
Nov 03 22:54:53.505568 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:54:53.505536    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931\": container with ID starting with a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931 not found: ID does not exist" containerID="a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931"
Nov 03 22:54:53.505732 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.505707    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931"} err="failed to get container status \"a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931\": rpc error: code = NotFound desc = could not find container \"a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931\": container with ID starting with a5fecfeb49410c95a4df50045abf54ce79c28d07b8d67103002b9ecad0829931 not found: ID does not exist"
Nov 03 22:54:53.652446 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.652412    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"]
Nov 03 22:54:53.703885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.703851    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr"]
Nov 03 22:54:53.774440 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.774346    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"]
Nov 03 22:54:53.778209 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.778183    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm"]
Nov 03 22:54:54.002413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:53.999384    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:54.056256 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:54.056185    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"]
Nov 03 22:54:54.056386 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:54.056371    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy" containerID="cri-o://ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17" gracePeriod=30
Nov 03 22:54:54.856045 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:54.856015    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" path="/var/lib/kubelet/pods/8826e5c8-7b53-4394-8f91-df31cab2db1d/volumes"
Nov 03 22:54:54.856601 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:54.856586    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" path="/var/lib/kubelet/pods/d7bd07ac-ea7b-4be4-8c87-35d96d7bce72/volumes"
Nov 03 22:54:57.795152 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:57.795115    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:59.180218 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.180109    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:54:59.343463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.343433    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:54:59.449350 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449321    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-kube-api-access-k55dn\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-kube-api-access-k55dn\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.449545 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449526    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-data\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.449590 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449574    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-envoy\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.449640 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449626    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-podinfo\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.449689 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449679    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-certs\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.449753 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449720    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40955271-29ca-4a4f-9c83-57ac8ab840de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40955271-29ca-4a4f-9c83-57ac8ab840de-istiod-ca-cert\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.449837 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449820    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-socket\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.449899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449838    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-data" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:59.449899 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449861    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-credential-socket\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.450017 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449902    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-token\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-token\") pod \"40955271-29ca-4a4f-9c83-57ac8ab840de\" (UID: \"40955271-29ca-4a4f-9c83-57ac8ab840de\") "
Nov 03 22:54:59.450017 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.449989    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-certs" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:59.450133 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450088    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-socket" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:59.450197 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450125    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-credential-socket" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:59.450494 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450451    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40955271-29ca-4a4f-9c83-57ac8ab840de-istiod-ca-cert" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:59.450646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450609    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40955271-29ca-4a4f-9c83-57ac8ab840de-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.450646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450635    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.450776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450652    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.450776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450667    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.450776 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.450681    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.451891 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.451851    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-kube-api-access-k55dn" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "kube-api-access-k55dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:59.452014 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.451945    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-envoy" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:59.452081 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.452047    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-podinfo" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:54:59.452302 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.452280    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-token" pod "40955271-29ca-4a4f-9c83-57ac8ab840de" (UID: "40955271-29ca-4a4f-9c83-57ac8ab840de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:59.527211 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.527172    5162 generic.go:356] "Generic (PLEG): container finished" podID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerID="ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17" exitCode=0
Nov 03 22:54:59.527422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.527370    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" event={"ID":"40955271-29ca-4a4f-9c83-57ac8ab840de","Type":"ContainerDied","Data":"ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17"}
Nov 03 22:54:59.527477 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.527456    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7" event={"ID":"40955271-29ca-4a4f-9c83-57ac8ab840de","Type":"ContainerDied","Data":"73327e3f93352d0562777a28944becb6e51d9a3d6401bfe01001158e498407f4"}
Nov 03 22:54:59.527510 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.527479    5162 scope.go:122] "RemoveContainer" containerID="ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17"
Nov 03 22:54:59.527605 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.527593    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"
Nov 03 22:54:59.542168 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.542148    5162 scope.go:122] "RemoveContainer" containerID="ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17"
Nov 03 22:54:59.542503 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:54:59.542484    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17\": container with ID starting with ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17 not found: ID does not exist" containerID="ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17"
Nov 03 22:54:59.542583 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.542522    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17"} err="failed to get container status \"ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17\": rpc error: code = NotFound desc = could not find container \"ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17\": container with ID starting with ed9abecebd1d0978493510abbeb379c8e2887baad1e5e5c176bddaa955bc8c17 not found: ID does not exist"
Nov 03 22:54:59.551967 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.551944    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k55dn\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-kube-api-access-k55dn\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.552061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.551973    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.552061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.551991    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.552061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.552005    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40955271-29ca-4a4f-9c83-57ac8ab840de-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:54:59.700368 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.700338    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"]
Nov 03 22:54:59.704023 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:54:59.703999    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7"]
Nov 03 22:55:00.856829 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:00.856797    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" path="/var/lib/kubelet/pods/40955271-29ca-4a4f-9c83-57ac8ab840de/volumes"
Nov 03 22:55:03.784234 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:03.784201    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:03.989207 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:03.989164    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:04.439472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:04.439427    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:04.640905 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:04.640877    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:08.141462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:08.140374    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:08.344549 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:08.344516    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:08.908467 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:08.908428    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:10.088567 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:55:10.088530    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:55:11.480898 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:11.480863    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:11.687815 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:11.687781    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:13.451557 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:13.451512    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:14.062741 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:14.062693    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:17.444056 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.444013    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:17.649465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.649342    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:17.800265 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800223    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"]
Nov 03 22:55:17.800914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800892    5162 cpu_manager.go:414] "containerMap: removing container" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy"
Nov 03 22:55:17.800914 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800913    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800927    5162 cpu_manager.go:414] "containerMap: removing container" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800934    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800953    5162 cpu_manager.go:414] "containerMap: removing container" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800960    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800990    5162 cpu_manager.go:414] "containerMap: removing container" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.800996    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.801027    5162 cpu_manager.go:414] "containerMap: removing container" podUID="132faa2b-51ce-4007-b4f4-dee77876a808" containerName="maas-api"
Nov 03 22:55:17.801091 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.801033    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="132faa2b-51ce-4007-b4f4-dee77876a808" containerName="maas-api"
Nov 03 22:55:17.801523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.801189    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="132faa2b-51ce-4007-b4f4-dee77876a808" containerName="maas-api"
Nov 03 22:55:17.801523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.801211    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="21fbf7f8-43dd-451b-973a-fdf8252cafa6" containerName="istio-proxy"
Nov 03 22:55:17.801523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.801222    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="8826e5c8-7b53-4394-8f91-df31cab2db1d" containerName="istio-proxy"
Nov 03 22:55:17.801523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.801233    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="40955271-29ca-4a4f-9c83-57ac8ab840de" containerName="istio-proxy"
Nov 03 22:55:17.801523 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.801244    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7bd07ac-ea7b-4be4-8c87-35d96d7bce72" containerName="istio-proxy"
Nov 03 22:55:17.804898 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.804876    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:17.858261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.858230    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"]
Nov 03 22:55:17.970412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.967921    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94tpk\" (UniqueName: \"kubernetes.io/projected/bb5c2a88-8196-495d-81f3-20bf038a60a6-kube-api-access-94tpk\") pod \"payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:17.970412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.967995    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5c2a88-8196-495d-81f3-20bf038a60a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:17.970412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.969310    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-12450a06-dockercfg-brfwg\""
Nov 03 22:55:17.970412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.969444    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-12450a06\""
Nov 03 22:55:17.987796 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.983654    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"]
Nov 03 22:55:17.989375 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:17.989354    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.029044 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.028994    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"]
Nov 03 22:55:18.070946 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.070909    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94tpk\" (UniqueName: \"kubernetes.io/projected/bb5c2a88-8196-495d-81f3-20bf038a60a6-kube-api-access-94tpk\") pod \"payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:18.071636 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.071605    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5c2a88-8196-495d-81f3-20bf038a60a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:18.071823 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.071805    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee85fab5-b858-4a48-b937-4a659e68580b-plugins-config-volume\") pod \"payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") " pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.072083 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.072067    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5tvr\" (UniqueName: \"kubernetes.io/projected/ee85fab5-b858-4a48-b937-4a659e68580b-kube-api-access-g5tvr\") pod \"payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") " pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.073189 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.073162    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5c2a88-8196-495d-81f3-20bf038a60a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:18.134428 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.134378    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94tpk\" (UniqueName: \"kubernetes.io/projected/bb5c2a88-8196-495d-81f3-20bf038a60a6-kube-api-access-94tpk\") pod \"payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:18.173180 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.173154    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee85fab5-b858-4a48-b937-4a659e68580b-plugins-config-volume\") pod \"payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") " pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.173309 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.173263    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5tvr\" (UniqueName: \"kubernetes.io/projected/ee85fab5-b858-4a48-b937-4a659e68580b-kube-api-access-g5tvr\") pod \"payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") " pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.173858 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.173835    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee85fab5-b858-4a48-b937-4a659e68580b-plugins-config-volume\") pod \"payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") " pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.235969 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.235908    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5tvr\" (UniqueName: \"kubernetes.io/projected/ee85fab5-b858-4a48-b937-4a659e68580b-kube-api-access-g5tvr\") pod \"payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") " pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.332972 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.332946    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:18.418118 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.418090    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:18.475187 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.475151    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:18.710949 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:55:18.710920    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee85fab5_b858_4a48_b937_4a659e68580b.slice/crio-17523914191b5891fa9d0f0bf89ee0da64a4add7f18f0cfe5b256afda76ab045 WatchSource:0}: Error finding container 17523914191b5891fa9d0f0bf89ee0da64a4add7f18f0cfe5b256afda76ab045: Status 404 returned error can't find the container with id 17523914191b5891fa9d0f0bf89ee0da64a4add7f18f0cfe5b256afda76ab045
Nov 03 22:55:18.715924 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.715896    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"]
Nov 03 22:55:18.806068 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:55:18.806044    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5c2a88_8196_495d_81f3_20bf038a60a6.slice/crio-d10c0ac8509f4fbae49a6fb75b24cab3313346adb289a6d236a5b41c4f2fe0ec WatchSource:0}: Error finding container d10c0ac8509f4fbae49a6fb75b24cab3313346adb289a6d236a5b41c4f2fe0ec: Status 404 returned error can't find the container with id d10c0ac8509f4fbae49a6fb75b24cab3313346adb289a6d236a5b41c4f2fe0ec
Nov 03 22:55:18.810750 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:18.810726    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"]
Nov 03 22:55:19.253642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.253474    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:19.700145 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.700105    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw" event={"ID":"ee85fab5-b858-4a48-b937-4a659e68580b","Type":"ContainerStarted","Data":"bcac9ce0c2d3001683283ec14a9181bc5d107850782c4f568bb0248d1901b102"}
Nov 03 22:55:19.700630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.700167    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:19.700630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.700189    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw" event={"ID":"ee85fab5-b858-4a48-b937-4a659e68580b","Type":"ContainerStarted","Data":"17523914191b5891fa9d0f0bf89ee0da64a4add7f18f0cfe5b256afda76ab045"}
Nov 03 22:55:19.701753 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.701731    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n" event={"ID":"bb5c2a88-8196-495d-81f3-20bf038a60a6","Type":"ContainerStarted","Data":"7d1776f69e714642f2e292bf23711d1bd6ed82ec30bffe62b400493ee90e3039"}
Nov 03 22:55:19.701859 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.701769    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n" event={"ID":"bb5c2a88-8196-495d-81f3-20bf038a60a6","Type":"ContainerStarted","Data":"d10c0ac8509f4fbae49a6fb75b24cab3313346adb289a6d236a5b41c4f2fe0ec"}
Nov 03 22:55:19.701859 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.701788    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:19.819658 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.819615    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw" podStartSLOduration=2.81960219 podStartE2EDuration="2.81960219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:55:19.81877096 +0000 UTC m=+1923.083470975" watchObservedRunningTime="2026-11-03 22:55:19.81960219 +0000 UTC m=+1923.084302200"
Nov 03 22:55:19.959472 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:19.953767    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n" podStartSLOduration=2.953745117 podStartE2EDuration="2.953745117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:55:19.945998004 +0000 UTC m=+1923.210698018" watchObservedRunningTime="2026-11-03 22:55:19.953745117 +0000 UTC m=+1923.218445133"
Nov 03 22:55:24.510345 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:24.510290    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:29.642774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:29.642733    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:29.849906 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:29.849838    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:30.055208 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:30.055168    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:30.710991 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:30.710961    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:55:30.711415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:30.711033    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:55:34.839202 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:34.839111    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:38.427417 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:38.425458    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:38.627454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:38.627386    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:39.995799 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:39.995760    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:40.089421 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:55:40.089372    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:55:45.146272 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:45.146238    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:50.384303 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:50.384261    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:50.622426 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:50.622354    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:50.877043 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:50.877009    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:55:55.660181 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:55:55.660134    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:00.433112 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:00.433067    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:04.149794 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:04.149760    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:05.297895 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:05.297856    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:10.118412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:10.115896    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:10.121150 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:56:10.121113    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:56:13.835460 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:13.835426    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:14.281213 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:14.281176    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:15.464877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:15.464830    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:15.842029 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:15.841952    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:16.147687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:16.147501    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:16.801617 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:16.801581    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:17.187517 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:17.187483    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:17.525486 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:17.521607    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:17.737717 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:17.737678    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:18.225868 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:18.225836    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:18.837166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:18.837127    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:22.567434 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:22.567366    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:22.778280 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:22.778247    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:22.977343 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:22.977310    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:23.327160 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:23.327072    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:25.089089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:25.089038    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:25.298844 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:25.298807    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:25.526965 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:25.526880    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:27.493225 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:27.493169    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:27.715869 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:27.715832    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:27.929749 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:27.929713    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:32.005515 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:32.005481    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:32.213521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:32.213488    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:32.443412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:32.442853    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:35.386662 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:35.386616    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:35.590760 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:35.590373    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:35.920514 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:35.919976    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:39.217116 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:39.216720    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:39.421969 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:39.421934    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:39.634314 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:39.634231    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:40.089386 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:56:40.089352    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:56:41.938682 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:41.938644    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:42.153951 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:42.153918    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:44.821798 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:44.821759    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:50.051783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:50.051741    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:52.274476 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:52.274428    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:55.464088 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:55.464051    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:59.072359 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:59.072321    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:56:59.278817 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:56:59.278687    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:00.676719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:00.676634    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:04.475831 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:04.475788    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:05.308345 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:05.308310    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:05.585323 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:05.585240    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:06.165063 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:06.165022    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:10.084660 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:57:10.084620    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:57:11.499920 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:11.496240    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:13.410734 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:13.410696    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:13.618707 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:13.618675    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:16.716019 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:16.715974    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:19.128955 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:19.128917    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:20.833480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:20.833442    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:21.891522 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:21.891360    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:25.593455 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.593419    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"]
Nov 03 22:57:25.598123 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.598095    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.703895 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.703852    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-12450a06-openshift-default-dockercfg-wvwz8\""
Nov 03 22:57:25.708547 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.708521    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"]
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.725750    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j57j4\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-kube-api-access-j57j4\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.725864    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-certs\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.725960    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-istio-token\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.726006    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eca86f5-f01a-4000-8118-27836d87ba36-istio-podinfo\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.726109    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-data\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.726163    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-envoy\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.726229    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-credential-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.726352    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eca86f5-f01a-4000-8118-27836d87ba36-istiod-ca-cert\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.728436 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.726411    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828001 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.827967    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-data\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828047    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-envoy\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828174 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828128    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-credential-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828262 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828232    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eca86f5-f01a-4000-8118-27836d87ba36-istiod-ca-cert\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828318 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828270    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828438    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j57j4\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-kube-api-access-j57j4\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828463 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828453    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-data\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828548    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-certs\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828555    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-credential-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828616 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828592    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828657    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-istio-token\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828700    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eca86f5-f01a-4000-8118-27836d87ba36-istio-podinfo\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.828783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.828740    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-certs\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.829067 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.829040    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eca86f5-f01a-4000-8118-27836d87ba36-istiod-ca-cert\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.830312 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.830291    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-envoy\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.830991 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.830972    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eca86f5-f01a-4000-8118-27836d87ba36-istio-podinfo\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.891364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.891296    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-istio-token\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.891506 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.891433    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j57j4\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-kube-api-access-j57j4\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-5l758\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:25.912061 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:25.912039    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:26.352823 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:57:26.352791    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eca86f5_f01a_4000_8118_27836d87ba36.slice/crio-ee639b71ebbf5031afed161530804cf4bb8f1c87185beacf0e2f57aac944dcff WatchSource:0}: Error finding container ee639b71ebbf5031afed161530804cf4bb8f1c87185beacf0e2f57aac944dcff: Status 404 returned error can't find the container with id ee639b71ebbf5031afed161530804cf4bb8f1c87185beacf0e2f57aac944dcff
Nov 03 22:57:26.355554 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.355517    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:57:26.355654 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.355605    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:57:26.355719 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.355651    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:57:26.356078 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.356054    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"]
Nov 03 22:57:26.797029 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.796994    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" event={"ID":"3eca86f5-f01a-4000-8118-27836d87ba36","Type":"ContainerStarted","Data":"dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3"}
Nov 03 22:57:26.797427 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.797049    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" event={"ID":"3eca86f5-f01a-4000-8118-27836d87ba36","Type":"ContainerStarted","Data":"ee639b71ebbf5031afed161530804cf4bb8f1c87185beacf0e2f57aac944dcff"}
Nov 03 22:57:26.912578 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.912542    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:26.928094 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:26.928040    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" podStartSLOduration=1.9280249189999998 podStartE2EDuration="1.928024919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:57:26.925821476 +0000 UTC m=+2050.190521487" watchObservedRunningTime="2026-11-03 22:57:26.928024919 +0000 UTC m=+2050.192724930"
Nov 03 22:57:27.158470 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:27.147912    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:27.914170 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:27.914136    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:57:27.914582 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:27.914188    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:57:28.913125 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:28.913092    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:57:28.913310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:28.913152    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:57:29.913318 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:29.913283    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:57:29.913744 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:29.913342    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:57:30.912566 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:30.912531    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:57:30.912744 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:30.912591    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:57:31.912963 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:31.912928    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:57:31.913422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:31.912987    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:57:32.372035 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:32.372003    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:32.917258 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:32.917229    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:32.917771 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:32.917637    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:32.918352 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:32.918333    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 22:57:37.678239 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:37.678199    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:40.085333 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:57:40.085298    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:57:40.729272 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.729242    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"]
Nov 03 22:57:40.734596 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.734576    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.812451 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.812416    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"]
Nov 03 22:57:40.855734 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.855704    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-credential-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.855928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.855749    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4pnz\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-kube-api-access-z4pnz\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.855928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.855856    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-data\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.855928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.855891    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istiod-ca-cert\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.855928 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.855921    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-token\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.856102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.855953    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-envoy\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.856102 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.856036    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-certs\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.856166 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.856128    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.856264 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.856253    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-podinfo\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.957959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.957925    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-data\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.957959 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.957959    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istiod-ca-cert\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958233 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.957980    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-token\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958233 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.957997    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-envoy\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958233 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958016    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-certs\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958233 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958114    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958328    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-podinfo\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958434    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-credential-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958519 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958471    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4pnz\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-kube-api-access-z4pnz\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958538    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-data\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958653 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958571    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-certs\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958730 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958669    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.958793 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.958773    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-credential-socket\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.959272 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.959250    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istiod-ca-cert\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.960459 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.960434    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-envoy\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:40.960857 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:40.960839    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-podinfo\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:41.041980 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.041914    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-token\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:41.042098 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.042035    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4pnz\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-kube-api-access-z4pnz\") pod \"e2e-mt-12450a06-openshift-default-7576d969f7-t5c69\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") " pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:41.051612 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.051584    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:41.176453 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.176416    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"]
Nov 03 22:57:41.177015 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.176983    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" containerID="cri-o://df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf" gracePeriod=30
Nov 03 22:57:41.307455 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.307363    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"]
Nov 03 22:57:41.307907 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.307856    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" containerID="cri-o://ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218" gracePeriod=30
Nov 03 22:57:41.482978 ip-10-0-143-0 kubenswrapper[5162]: W1103 22:57:41.482950    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d WatchSource:0}: Error finding container 339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d: Status 404 returned error can't find the container with id 339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d
Nov 03 22:57:41.485772 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.485735    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:57:41.485885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.485822    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:57:41.485885 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.485869    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 22:57:41.486051 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.485908    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"]
Nov 03 22:57:41.627956 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.627866    5162 patch_prober.go:29] interesting pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:57:41.627956 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.627925    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:57:41.900147 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.900060    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" event={"ID":"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06","Type":"ContainerStarted","Data":"3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147"}
Nov 03 22:57:41.900147 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:41.900141    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" event={"ID":"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06","Type":"ContainerStarted","Data":"339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d"}
Nov 03 22:57:42.026023 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:42.025969    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podStartSLOduration=2.025956311 podStartE2EDuration="2.025956311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:57:42.02356141 +0000 UTC m=+2065.288261420" watchObservedRunningTime="2026-11-03 22:57:42.025956311 +0000 UTC m=+2065.290656321"
Nov 03 22:57:42.061416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:42.056458    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:42.065413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:42.063537    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 03 22:57:42.065413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:42.063606    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 03 22:57:42.969069 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:42.965204    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:43.005146 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:43.005045    5162 patch_prober.go:29] interesting pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:57:43.005342 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:43.005145    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:57:43.056419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:43.055180    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 03 22:57:43.056419 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:43.055234    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 03 22:57:44.065412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:44.064481    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 03 22:57:44.065412 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:44.064542    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 03 22:57:45.063416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:45.062479    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 03 22:57:45.063416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:45.062541    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 03 22:57:46.065425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.061698    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 03 22:57:46.065425 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.061757    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 03 22:57:46.504462 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.504439    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:57:46.578373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578287    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-socket\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578373 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578340    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istiod-ca-cert\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578399    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-certs\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578445    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-envoy\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578542    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-data\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578601    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-token\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-token\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578825 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578635    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-kube-api-access-jsffg\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-kube-api-access-jsffg\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578825 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578712    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-podinfo\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.578825 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.578817    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-credential-socket\") pod \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\" (UID: \"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d\") "
Nov 03 22:57:46.580610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.580339    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-socket" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.580610 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.580566    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-certs" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.580923 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.580889    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istiod-ca-cert" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:57:46.581022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.580953    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-data" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.581022 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.581002    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-credential-socket" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.585886 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.585849    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-token" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:46.587080 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.587044    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-podinfo" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:57:46.587240 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.587217    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-envoy" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.589499 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.589476    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-kube-api-access-jsffg" pod "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" (UID: "3c8f4d07-e09c-40dd-8370-a15b6dc5c26d"). InnerVolumeSpecName "kube-api-access-jsffg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:46.624990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.624968    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:57:46.680088 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680057    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-socket\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680100    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-certs\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680257 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680230    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-token\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-token\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680364    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-socket" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.680468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680383    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-certs" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.680468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680382    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-credential-socket\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680468 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680443    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-kube-api-access-jxz74\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-kube-api-access-jxz74\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680629 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680479    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-podinfo\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680629 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680521    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-data\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680629 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680596    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-envoy\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680635    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-credential-socket" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.680802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680660    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istiod-ca-cert\") pod \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\" (UID: \"f12f2950-90f4-4c21-97ab-ef5a5541d9ab\") "
Nov 03 22:57:46.680802 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.680760    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-data" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.681220 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681178    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istiod-ca-cert" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:57:46.681445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681418    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681444    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681458    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681470    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681482    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681494    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681507    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681520    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681532    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681544    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681557    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsffg\" (UniqueName: \"kubernetes.io/projected/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-kube-api-access-jsffg\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681570    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681584    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.681615 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.681600    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.683528 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.683477    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-podinfo" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:57:46.684125 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.684089    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-token" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:46.684219 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.684201    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-kube-api-access-jxz74" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "kube-api-access-jxz74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:46.684361 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.684323    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-envoy" pod "f12f2950-90f4-4c21-97ab-ef5a5541d9ab" (UID: "f12f2950-90f4-4c21-97ab-ef5a5541d9ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:46.782620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.782586    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxz74\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-kube-api-access-jxz74\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.782620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.782615    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.782620 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.782624    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.782862 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.782632    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12f2950-90f4-4c21-97ab-ef5a5541d9ab-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:57:46.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.962902    5162 generic.go:356] "Generic (PLEG): container finished" podID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerID="ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218" exitCode=0
Nov 03 22:57:46.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.963166    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" event={"ID":"f12f2950-90f4-4c21-97ab-ef5a5541d9ab","Type":"ContainerDied","Data":"ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218"}
Nov 03 22:57:46.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.963217    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4" event={"ID":"f12f2950-90f4-4c21-97ab-ef5a5541d9ab","Type":"ContainerDied","Data":"ebace35289a3a9cf35bd021a9b91c3346eae04c53df0e468429d83d3068882d1"}
Nov 03 22:57:46.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.963247    5162 scope.go:122] "RemoveContainer" containerID="ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218"
Nov 03 22:57:46.965415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.963449    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"
Nov 03 22:57:46.967884 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.967861    5162 generic.go:356] "Generic (PLEG): container finished" podID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerID="df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf" exitCode=0
Nov 03 22:57:46.968011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.967929    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"
Nov 03 22:57:46.968011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.967950    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" event={"ID":"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d","Type":"ContainerDied","Data":"df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf"}
Nov 03 22:57:46.968727 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.968036    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t" event={"ID":"3c8f4d07-e09c-40dd-8370-a15b6dc5c26d","Type":"ContainerDied","Data":"512bd3959b070e21e76dde6d151036ae06a8702096f81d2538e584350a45b83c"}
Nov 03 22:57:46.977544 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.977527    5162 scope.go:122] "RemoveContainer" containerID="ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218"
Nov 03 22:57:46.977888 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:57:46.977863    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218\": container with ID starting with ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218 not found: ID does not exist" containerID="ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218"
Nov 03 22:57:46.977954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.977900    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218"} err="failed to get container status \"ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218\": rpc error: code = NotFound desc = could not find container \"ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218\": container with ID starting with ca701ff843b908135549c3b957518aa18f911c177c9ca249a16e0395bf837218 not found: ID does not exist"
Nov 03 22:57:46.977954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.977924    5162 scope.go:122] "RemoveContainer" containerID="df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf"
Nov 03 22:57:46.987952 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.987927    5162 scope.go:122] "RemoveContainer" containerID="df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf"
Nov 03 22:57:46.988261 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:57:46.988238    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf\": container with ID starting with df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf not found: ID does not exist" containerID="df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf"
Nov 03 22:57:46.988357 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:46.988269    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf"} err="failed to get container status \"df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf\": rpc error: code = NotFound desc = could not find container \"df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf\": container with ID starting with df2d328645ccbd214c74128d2bd6f53cd3256d2d7bfdd069496189443490c2cf not found: ID does not exist"
Nov 03 22:57:47.059764 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.059732    5162 patch_prober.go:29] interesting pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 03 22:57:47.060020 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.059994    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 03 22:57:47.137937 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.137858    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"]
Nov 03 22:57:47.162841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.162805    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t"]
Nov 03 22:57:47.271013 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.270971    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"]
Nov 03 22:57:47.274307 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.274283    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4"]
Nov 03 22:57:47.741286 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.741233    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:47.995791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:47.995711    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:48.057261 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:48.057236    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:48.057655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:48.057640    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:48.058766 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:48.058746    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 22:57:48.199502 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:48.199467    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:48.855332 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:48.855298    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" path="/var/lib/kubelet/pods/3c8f4d07-e09c-40dd-8370-a15b6dc5c26d/volumes"
Nov 03 22:57:48.855906 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:48.855887    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" path="/var/lib/kubelet/pods/f12f2950-90f4-4c21-97ab-ef5a5541d9ab/volumes"
Nov 03 22:57:53.365571 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:53.365533    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:53.623822 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:53.623733    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:57:58.433850 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:57:58.431984    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:01.194165 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:01.194129    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:01.402783 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:01.402737    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:02.635890 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:02.635852    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:04.140807 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:04.140776    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:04.403460 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:04.403370    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:04.612646 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:04.612589    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:04.832739 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:04.832709    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:05.034662 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:05.034626    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:05.254193 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:05.254160    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:09.790993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:09.790899    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:10.086258 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:58:10.086163    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:58:11.354223 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:11.354185    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:11.623186 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:11.623096    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:11.838426 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:11.838329    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:12.645952 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:12.645913    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:12.873841 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:12.873801    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:14.323244 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:14.323205    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:14.592971 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:14.592873    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:15.047416 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:15.040770    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:19.204301 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:19.204261    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:19.532156 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:19.532071    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:19.817876 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:19.817757    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:21.143119 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:21.143076    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:21.470663 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:21.470628    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:21.712518 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:21.712482    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:22.162900 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:22.162864    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:23.888944 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:23.888909    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"]
Nov 03 22:58:24.006977 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:24.006944    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"]
Nov 03 22:58:24.334066 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:24.334027    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:24.536650 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:24.536618    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:24.738947 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:24.738907    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:25.384415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:25.371293    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:25.585809 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:25.585772    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:26.710112 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:26.710073    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:28.766954 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:28.766909    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:28.889365 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:28.889312    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw" podUID="ee85fab5-b858-4a48-b937-4a659e68580b" containerName="payload-processing" containerID="cri-o://bcac9ce0c2d3001683283ec14a9181bc5d107850782c4f568bb0248d1901b102" gracePeriod=25
Nov 03 22:58:28.993795 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:28.993758    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:29.008686 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.008652    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n" podUID="bb5c2a88-8196-495d-81f3-20bf038a60a6" containerName="payload-pre-processing" containerID="cri-o://7d1776f69e714642f2e292bf23711d1bd6ed82ec30bffe62b400493ee90e3039" gracePeriod=25
Nov 03 22:58:29.270669 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.269966    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:29.319992 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.319956    5162 generic.go:356] "Generic (PLEG): container finished" podID="ee85fab5-b858-4a48-b937-4a659e68580b" containerID="bcac9ce0c2d3001683283ec14a9181bc5d107850782c4f568bb0248d1901b102" exitCode=0
Nov 03 22:58:29.320155 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.320098    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw" event={"ID":"ee85fab5-b858-4a48-b937-4a659e68580b","Type":"ContainerDied","Data":"bcac9ce0c2d3001683283ec14a9181bc5d107850782c4f568bb0248d1901b102"}
Nov 03 22:58:29.320155 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.320140    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw" event={"ID":"ee85fab5-b858-4a48-b937-4a659e68580b","Type":"ContainerDied","Data":"17523914191b5891fa9d0f0bf89ee0da64a4add7f18f0cfe5b256afda76ab045"}
Nov 03 22:58:29.320270 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.320157    5162 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17523914191b5891fa9d0f0bf89ee0da64a4add7f18f0cfe5b256afda76ab045"
Nov 03 22:58:29.323292 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.323259    5162 generic.go:356] "Generic (PLEG): container finished" podID="bb5c2a88-8196-495d-81f3-20bf038a60a6" containerID="7d1776f69e714642f2e292bf23711d1bd6ed82ec30bffe62b400493ee90e3039" exitCode=0
Nov 03 22:58:29.323480 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.323361    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n" event={"ID":"bb5c2a88-8196-495d-81f3-20bf038a60a6","Type":"ContainerDied","Data":"7d1776f69e714642f2e292bf23711d1bd6ed82ec30bffe62b400493ee90e3039"}
Nov 03 22:58:29.330566 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.330535    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:58:29.419825 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.419793    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee85fab5-b858-4a48-b937-4a659e68580b-kube-api-access-g5tvr\" (UniqueName: \"kubernetes.io/projected/ee85fab5-b858-4a48-b937-4a659e68580b-kube-api-access-g5tvr\") pod \"ee85fab5-b858-4a48-b937-4a659e68580b\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") "
Nov 03 22:58:29.419990 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.419920    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee85fab5-b858-4a48-b937-4a659e68580b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee85fab5-b858-4a48-b937-4a659e68580b-plugins-config-volume\") pod \"ee85fab5-b858-4a48-b937-4a659e68580b\" (UID: \"ee85fab5-b858-4a48-b937-4a659e68580b\") "
Nov 03 22:58:29.420310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.420279    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee85fab5-b858-4a48-b937-4a659e68580b-plugins-config-volume" pod "ee85fab5-b858-4a48-b937-4a659e68580b" (UID: "ee85fab5-b858-4a48-b937-4a659e68580b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:29.420630 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.420609    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee85fab5-b858-4a48-b937-4a659e68580b-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.421985 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.421957    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee85fab5-b858-4a48-b937-4a659e68580b-kube-api-access-g5tvr" pod "ee85fab5-b858-4a48-b937-4a659e68580b" (UID: "ee85fab5-b858-4a48-b937-4a659e68580b"). InnerVolumeSpecName "kube-api-access-g5tvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:29.444955 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.444913    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:58:29.521837 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.521801    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb5c2a88-8196-495d-81f3-20bf038a60a6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5c2a88-8196-495d-81f3-20bf038a60a6-plugins-config-volume\") pod \"bb5c2a88-8196-495d-81f3-20bf038a60a6\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") "
Nov 03 22:58:29.522055 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.522018    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb5c2a88-8196-495d-81f3-20bf038a60a6-kube-api-access-94tpk\" (UniqueName: \"kubernetes.io/projected/bb5c2a88-8196-495d-81f3-20bf038a60a6-kube-api-access-94tpk\") pod \"bb5c2a88-8196-495d-81f3-20bf038a60a6\" (UID: \"bb5c2a88-8196-495d-81f3-20bf038a60a6\") "
Nov 03 22:58:29.522353 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.522324    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5c2a88-8196-495d-81f3-20bf038a60a6-plugins-config-volume" pod "bb5c2a88-8196-495d-81f3-20bf038a60a6" (UID: "bb5c2a88-8196-495d-81f3-20bf038a60a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:29.522747 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.522724    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5tvr\" (UniqueName: \"kubernetes.io/projected/ee85fab5-b858-4a48-b937-4a659e68580b-kube-api-access-g5tvr\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.522820 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.522755    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb5c2a88-8196-495d-81f3-20bf038a60a6-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.524958 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.524931    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5c2a88-8196-495d-81f3-20bf038a60a6-kube-api-access-94tpk" pod "bb5c2a88-8196-495d-81f3-20bf038a60a6" (UID: "bb5c2a88-8196-495d-81f3-20bf038a60a6"). InnerVolumeSpecName "kube-api-access-94tpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:29.624454 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.624316    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94tpk\" (UniqueName: \"kubernetes.io/projected/bb5c2a88-8196-495d-81f3-20bf038a60a6-kube-api-access-94tpk\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.980750 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:29.980714    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:30.332592 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.332544    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n" event={"ID":"bb5c2a88-8196-495d-81f3-20bf038a60a6","Type":"ContainerDied","Data":"d10c0ac8509f4fbae49a6fb75b24cab3313346adb289a6d236a5b41c4f2fe0ec"}
Nov 03 22:58:30.332791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.332622    5162 scope.go:122] "RemoveContainer" containerID="7d1776f69e714642f2e292bf23711d1bd6ed82ec30bffe62b400493ee90e3039"
Nov 03 22:58:30.332791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.332571    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"
Nov 03 22:58:30.332791 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.332572    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"
Nov 03 22:58:30.590487 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.583790    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"]
Nov 03 22:58:30.605415 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.600936    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw"]
Nov 03 22:58:30.661028 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.660951    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:30.714114 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.714071    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"]
Nov 03 22:58:30.722413 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.721236    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n"]
Nov 03 22:58:30.861458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.859473    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb5c2a88-8196-495d-81f3-20bf038a60a6" path="/var/lib/kubelet/pods/bb5c2a88-8196-495d-81f3-20bf038a60a6/volumes"
Nov 03 22:58:30.861458 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.860320    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee85fab5-b858-4a48-b937-4a659e68580b" path="/var/lib/kubelet/pods/ee85fab5-b858-4a48-b937-4a659e68580b/volumes"
Nov 03 22:58:30.929543 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:30.929507    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:31.214364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:31.214327    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:32.358040 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:32.357999    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:32.760115 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:32.754936    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:32.956445 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:32.956374    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:35.337642 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:35.337603    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:35.557048 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:35.557009    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:35.781792 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:35.781757    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:37.049563 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:37.049512    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:37.492965 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:37.492928    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:38.564364 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:38.564324    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:38.828846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:38.828754    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:39.038437 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:39.036954    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:39.295422 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:39.293158    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:39.497464 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:39.497429    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:40.086910 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:58:40.086877    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:58:40.091970 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:40.091939    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:42.968204 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:42.968166    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:44.786150 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:44.786083    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:46.392918 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:46.392867    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:46.592538 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:46.592495    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:46.804941 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:46.804907    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:49.965947 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:49.965904    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:52.046846 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:52.046810    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:52.253993 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:52.253954    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:53.782288 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:53.782236    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:54.034051 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:54.033960    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:55.121774 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:55.121731    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:58.812641 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:58.812608    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"]
Nov 03 22:58:58.924957 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:58.924923    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"]
Nov 03 22:58:59.258724 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:59.258688    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:59.469011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:59.468973    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:58:59.936763 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:58:59.936729    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:03.812921 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:03.812865    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl" podUID="3a5f9047-afa4-4610-bcb3-866aac6cfe00" containerName="payload-processing" containerID="cri-o://2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13" gracePeriod=25
Nov 03 22:59:03.925655 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:03.925616    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx" podUID="27ba7a45-61ff-4f1a-aaba-709ee7c8839a" containerName="payload-pre-processing" containerID="cri-o://2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48" gracePeriod=25
Nov 03 22:59:04.085671 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.085596    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:59:04.205536 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.205509    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a5f9047-afa4-4610-bcb3-866aac6cfe00-kube-api-access-57v5l\" (UniqueName: \"kubernetes.io/projected/3a5f9047-afa4-4610-bcb3-866aac6cfe00-kube-api-access-57v5l\") pod \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") "
Nov 03 22:59:04.205706 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.205658    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a5f9047-afa4-4610-bcb3-866aac6cfe00-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a5f9047-afa4-4610-bcb3-866aac6cfe00-plugins-config-volume\") pod \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\" (UID: \"3a5f9047-afa4-4610-bcb3-866aac6cfe00\") "
Nov 03 22:59:04.206252 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.206225    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a5f9047-afa4-4610-bcb3-866aac6cfe00-plugins-config-volume" pod "3a5f9047-afa4-4610-bcb3-866aac6cfe00" (UID: "3a5f9047-afa4-4610-bcb3-866aac6cfe00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:59:04.206587 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.206564    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a5f9047-afa4-4610-bcb3-866aac6cfe00-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:59:04.210441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.210133    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:59:04.210441 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.210251    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a5f9047-afa4-4610-bcb3-866aac6cfe00-kube-api-access-57v5l" pod "3a5f9047-afa4-4610-bcb3-866aac6cfe00" (UID: "3a5f9047-afa4-4610-bcb3-866aac6cfe00"). InnerVolumeSpecName "kube-api-access-57v5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:59:04.307260 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.307227    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-plugins-config-volume\") pod \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") "
Nov 03 22:59:04.307466 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.307445    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-kube-api-access-tzkql\" (UniqueName: \"kubernetes.io/projected/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-kube-api-access-tzkql\") pod \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\" (UID: \"27ba7a45-61ff-4f1a-aaba-709ee7c8839a\") "
Nov 03 22:59:04.307718 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.307691    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-plugins-config-volume" pod "27ba7a45-61ff-4f1a-aaba-709ee7c8839a" (UID: "27ba7a45-61ff-4f1a-aaba-709ee7c8839a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:59:04.307941 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.307923    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57v5l\" (UniqueName: \"kubernetes.io/projected/3a5f9047-afa4-4610-bcb3-866aac6cfe00-kube-api-access-57v5l\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:59:04.308029 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.307943    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:59:04.309581 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.309559    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-kube-api-access-tzkql" pod "27ba7a45-61ff-4f1a-aaba-709ee7c8839a" (UID: "27ba7a45-61ff-4f1a-aaba-709ee7c8839a"). InnerVolumeSpecName "kube-api-access-tzkql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:59:04.409421 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.409315    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzkql\" (UniqueName: \"kubernetes.io/projected/27ba7a45-61ff-4f1a-aaba-709ee7c8839a-kube-api-access-tzkql\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 22:59:04.615595 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.615563    5162 generic.go:356] "Generic (PLEG): container finished" podID="3a5f9047-afa4-4610-bcb3-866aac6cfe00" containerID="2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13" exitCode=0
Nov 03 22:59:04.615800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.615611    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"
Nov 03 22:59:04.615800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.615641    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl" event={"ID":"3a5f9047-afa4-4610-bcb3-866aac6cfe00","Type":"ContainerDied","Data":"2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13"}
Nov 03 22:59:04.615800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.615696    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl" event={"ID":"3a5f9047-afa4-4610-bcb3-866aac6cfe00","Type":"ContainerDied","Data":"d138584ce5b6b2f4d149df8ef5093137258cdefeb0c96f67f4a88057d00d9501"}
Nov 03 22:59:04.615800 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.615726    5162 scope.go:122] "RemoveContainer" containerID="2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13"
Nov 03 22:59:04.617176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.617147    5162 generic.go:356] "Generic (PLEG): container finished" podID="27ba7a45-61ff-4f1a-aaba-709ee7c8839a" containerID="2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48" exitCode=0
Nov 03 22:59:04.617293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.617196    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"
Nov 03 22:59:04.617293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.617227    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx" event={"ID":"27ba7a45-61ff-4f1a-aaba-709ee7c8839a","Type":"ContainerDied","Data":"2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48"}
Nov 03 22:59:04.617293 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.617269    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx" event={"ID":"27ba7a45-61ff-4f1a-aaba-709ee7c8839a","Type":"ContainerDied","Data":"e1b2662b5473166057b0629477beb704ffd0b3a019e8dcfc658c5dbb45f58377"}
Nov 03 22:59:04.628071 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.628042    5162 scope.go:122] "RemoveContainer" containerID="2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13"
Nov 03 22:59:04.628779 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:59:04.628752    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13\": container with ID starting with 2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13 not found: ID does not exist" containerID="2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13"
Nov 03 22:59:04.628880 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.628790    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13"} err="failed to get container status \"2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13\": rpc error: code = NotFound desc = could not find container \"2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13\": container with ID starting with 2ba73ef8c08dbe8d9f801a933cf22ee1ac23876a9e2d63d1f9c07bd7017e2d13 not found: ID does not exist"
Nov 03 22:59:04.628880 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.628814    5162 scope.go:122] "RemoveContainer" containerID="2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48"
Nov 03 22:59:04.643360 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.643339    5162 scope.go:122] "RemoveContainer" containerID="2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48"
Nov 03 22:59:04.643708 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:59:04.643681    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48\": container with ID starting with 2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48 not found: ID does not exist" containerID="2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48"
Nov 03 22:59:04.643759 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.643718    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48"} err="failed to get container status \"2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48\": rpc error: code = NotFound desc = could not find container \"2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48\": container with ID starting with 2c008a30eae1a87e815660de2dbe49c8a80b0b76db7c4b22ff6031d852f86b48 not found: ID does not exist"
Nov 03 22:59:04.788699 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.788664    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"]
Nov 03 22:59:04.794337 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.794313    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl"]
Nov 03 22:59:04.829811 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.829779    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:04.856489 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.856463    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a5f9047-afa4-4610-bcb3-866aac6cfe00" path="/var/lib/kubelet/pods/3a5f9047-afa4-4610-bcb3-866aac6cfe00/volumes"
Nov 03 22:59:04.907426 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.907352    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"]
Nov 03 22:59:04.912176 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:04.912146    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx"]
Nov 03 22:59:05.176310 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:05.176279    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:06.855481 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:06.855450    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27ba7a45-61ff-4f1a-aaba-709ee7c8839a" path="/var/lib/kubelet/pods/27ba7a45-61ff-4f1a-aaba-709ee7c8839a/volumes"
Nov 03 22:59:09.571429 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:09.571353    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:10.084636 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:59:10.084597    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:59:14.543697 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:14.543661    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:18.744592 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:18.744556    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:19.562465 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:19.562424    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:20.296046 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:20.296008    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:20.524032 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:20.523995    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:20.726011 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:20.725975    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:21.300383 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:21.300346    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:25.798983 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:25.795414    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:26.267807 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:26.267766    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:26.472090 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:26.472053    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:26.727451 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:26.727421    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:28.236363 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:28.234308    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:28.477687 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:28.477648    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:31.254629 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:31.254589    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:32.896940 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:32.896906    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:33.156317 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:33.156239    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:36.821101 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:36.821064    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:37.447877 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:37.447792    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:37.653521 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:37.653451    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:39.702219 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:39.702181    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:39.911089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:39.911052    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:40.086897 ip-10-0-143-0 kubenswrapper[5162]: E1103 22:59:40.086809    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 22:59:42.218930 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:42.218894    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:46.669706 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:46.669671    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:46.954140 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:46.954054    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:48.985686 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:48.985653    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:49.190340 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:49.190299    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:52.840443 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:52.840340    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:53.061861 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:53.061814    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:53.303089 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:53.303055    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:54.623337 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:54.623008    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:54.827613 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:54.827573    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:55.034385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:55.034345    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 22:59:59.994385 ip-10-0-143-0 kubenswrapper[5162]: I1103 22:59:59.994326    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:00.215739 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.215710    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"]
Nov 03 23:00:00.216304 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216288    5162 cpu_manager.go:414] "containerMap: removing container" podUID="27ba7a45-61ff-4f1a-aaba-709ee7c8839a" containerName="payload-pre-processing"
Nov 03 23:00:00.216351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216306    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27ba7a45-61ff-4f1a-aaba-709ee7c8839a" containerName="payload-pre-processing"
Nov 03 23:00:00.216351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216322    5162 cpu_manager.go:414] "containerMap: removing container" podUID="bb5c2a88-8196-495d-81f3-20bf038a60a6" containerName="payload-pre-processing"
Nov 03 23:00:00.216351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216327    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb5c2a88-8196-495d-81f3-20bf038a60a6" containerName="payload-pre-processing"
Nov 03 23:00:00.216351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216344    5162 cpu_manager.go:414] "containerMap: removing container" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy"
Nov 03 23:00:00.216351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216349    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216363    5162 cpu_manager.go:414] "containerMap: removing container" podUID="3a5f9047-afa4-4610-bcb3-866aac6cfe00" containerName="payload-processing"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216368    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a5f9047-afa4-4610-bcb3-866aac6cfe00" containerName="payload-processing"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216405    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216410    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216423    5162 cpu_manager.go:414] "containerMap: removing container" podUID="ee85fab5-b858-4a48-b937-4a659e68580b" containerName="payload-processing"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216428    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee85fab5-b858-4a48-b937-4a659e68580b" containerName="payload-processing"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216517    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f12f2950-90f4-4c21-97ab-ef5a5541d9ab" containerName="istio-proxy"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216526    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee85fab5-b858-4a48-b937-4a659e68580b" containerName="payload-processing"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216532    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="27ba7a45-61ff-4f1a-aaba-709ee7c8839a" containerName="payload-pre-processing"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216539    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb5c2a88-8196-495d-81f3-20bf038a60a6" containerName="payload-pre-processing"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216548    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c8f4d07-e09c-40dd-8370-a15b6dc5c26d" containerName="istio-proxy"
Nov 03 23:00:00.216551 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.216557    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a5f9047-afa4-4610-bcb3-866aac6cfe00" containerName="payload-processing"
Nov 03 23:00:00.219902 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.219884    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"]
Nov 03 23:00:00.220017 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.219997    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.223891 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.223868    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.242376 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.242355    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"]
Nov 03 23:00:00.246055 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.245999    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.267361 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.267335    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"]
Nov 03 23:00:00.271473 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.271455    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"]
Nov 03 23:00:00.271588 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.271569    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.275898 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.275874    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.312987 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.312951    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4xb6\" (UniqueName: \"kubernetes.io/projected/a876d4eb-4282-4ae9-9726-63a18708508f-kube-api-access-d4xb6\") pod \"maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.313197 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.313020    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cf17ac-f1dd-44dd-b3bb-627c61227327-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.313197 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.313097    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfk7x\" (UniqueName: \"kubernetes.io/projected/f8cf17ac-f1dd-44dd-b3bb-627c61227327-kube-api-access-rfk7x\") pod \"maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.313197 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.313160    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.313518 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.313209    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcsss\" (UniqueName: \"kubernetes.io/projected/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-kube-api-access-tcsss\") pod \"maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.313518 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.313254    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjm78\" (UniqueName: \"kubernetes.io/projected/bbdb117f-e199-4424-9674-69c4f71224d4-kube-api-access-sjm78\") pod \"maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.313518 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.313290    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdb117f-e199-4424-9674-69c4f71224d4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.313518 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.313380    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a876d4eb-4282-4ae9-9726-63a18708508f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.329193 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.329164    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"]
Nov 03 23:00:00.335568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.335533    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjll9\""
Nov 03 23:00:00.363150 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.363110    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"]
Nov 03 23:00:00.372708 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.372678    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"]
Nov 03 23:00:00.376977 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.376946    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"]
Nov 03 23:00:00.379874 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.379850    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"]
Nov 03 23:00:00.414735 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.414711    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a876d4eb-4282-4ae9-9726-63a18708508f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.414832 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.414749    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb7vm\" (UniqueName: \"kubernetes.io/projected/9f7b63bc-125d-4b96-b401-c4dfcc396922-kube-api-access-sb7vm\") pod \"maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.414832 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.414822    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4xb6\" (UniqueName: \"kubernetes.io/projected/a876d4eb-4282-4ae9-9726-63a18708508f-kube-api-access-d4xb6\") pod \"maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.414913 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.414848    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7b63bc-125d-4b96-b401-c4dfcc396922-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.414952 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.414914    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cf17ac-f1dd-44dd-b3bb-627c61227327-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.415088 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415068    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfk7x\" (UniqueName: \"kubernetes.io/projected/f8cf17ac-f1dd-44dd-b3bb-627c61227327-kube-api-access-rfk7x\") pod \"maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.415165 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415150    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.415224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415206    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcsss\" (UniqueName: \"kubernetes.io/projected/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-kube-api-access-tcsss\") pod \"maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.415291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415261    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjm78\" (UniqueName: \"kubernetes.io/projected/bbdb117f-e199-4424-9674-69c4f71224d4-kube-api-access-sjm78\") pod \"maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.415351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415299    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdb117f-e199-4424-9674-69c4f71224d4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.415546 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415525    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a876d4eb-4282-4ae9-9726-63a18708508f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.415725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415703    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cf17ac-f1dd-44dd-b3bb-627c61227327-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.415844 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415827    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.415980 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.415961    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdb117f-e199-4424-9674-69c4f71224d4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.478056 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.478020    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfk7x\" (UniqueName: \"kubernetes.io/projected/f8cf17ac-f1dd-44dd-b3bb-627c61227327-kube-api-access-rfk7x\") pod \"maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.478218 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.478114    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjm78\" (UniqueName: \"kubernetes.io/projected/bbdb117f-e199-4424-9674-69c4f71224d4-kube-api-access-sjm78\") pod \"maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.478266 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.478212    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcsss\" (UniqueName: \"kubernetes.io/projected/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-kube-api-access-tcsss\") pod \"maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.478306 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.478271    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4xb6\" (UniqueName: \"kubernetes.io/projected/a876d4eb-4282-4ae9-9726-63a18708508f-kube-api-access-d4xb6\") pod \"maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.516661 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.516592    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb7vm\" (UniqueName: \"kubernetes.io/projected/9f7b63bc-125d-4b96-b401-c4dfcc396922-kube-api-access-sb7vm\") pod \"maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.516802 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.516722    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7b63bc-125d-4b96-b401-c4dfcc396922-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.517292 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.517269    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7b63bc-125d-4b96-b401-c4dfcc396922-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.535963 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.535936    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:00.542296 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.542277    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:00.561771 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.561747    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:00.582555 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.582532    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb7vm\" (UniqueName: \"kubernetes.io/projected/9f7b63bc-125d-4b96-b401-c4dfcc396922-kube-api-access-sb7vm\") pod \"maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.586879 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.586855    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:00.596370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.596187    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:00.932706 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:00:00.932676    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda876d4eb_4282_4ae9_9726_63a18708508f.slice/crio-b10139e03ac2243ca25583726f2066837ce98aacfd02208bcbf211604de6eba6 WatchSource:0}: Error finding container b10139e03ac2243ca25583726f2066837ce98aacfd02208bcbf211604de6eba6: Status 404 returned error can't find the container with id b10139e03ac2243ca25583726f2066837ce98aacfd02208bcbf211604de6eba6
Nov 03 23:00:00.936644 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.936623    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"]
Nov 03 23:00:00.979262 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:00:00.979229    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487 WatchSource:0}: Error finding container 508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487: Status 404 returned error can't find the container with id 508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487
Nov 03 23:00:00.983027 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:00:00.982987    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb8a3d7_b5fa_4761_98f3_065f9365d34a.slice/crio-a315ee0d3faba843ba544b68adabbfd3bc15efce471d1fc876c13c5cdaf6b7e4 WatchSource:0}: Error finding container a315ee0d3faba843ba544b68adabbfd3bc15efce471d1fc876c13c5cdaf6b7e4: Status 404 returned error can't find the container with id a315ee0d3faba843ba544b68adabbfd3bc15efce471d1fc876c13c5cdaf6b7e4
Nov 03 23:00:00.984111 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.984085    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"]
Nov 03 23:00:00.988206 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:00.988184    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"]
Nov 03 23:00:01.029521 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:00:01.029492    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdb117f_e199_4424_9674_69c4f71224d4.slice/crio-fa4032a40a92252764dd691e6b7d624d2380f949d7e3edd577053d84037384b8 WatchSource:0}: Error finding container fa4032a40a92252764dd691e6b7d624d2380f949d7e3edd577053d84037384b8: Status 404 returned error can't find the container with id fa4032a40a92252764dd691e6b7d624d2380f949d7e3edd577053d84037384b8
Nov 03 23:00:01.034141 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:01.034110    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"]
Nov 03 23:00:01.048109 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:00:01.048080    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7b63bc_125d_4b96_b401_c4dfcc396922.slice/crio-6572c5c7c55e7ec114e7abe1d19f5acc4fb30d193f84712b5d3a82f2be41e8e0 WatchSource:0}: Error finding container 6572c5c7c55e7ec114e7abe1d19f5acc4fb30d193f84712b5d3a82f2be41e8e0: Status 404 returned error can't find the container with id 6572c5c7c55e7ec114e7abe1d19f5acc4fb30d193f84712b5d3a82f2be41e8e0
Nov 03 23:00:01.048517 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:01.048491    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"]
Nov 03 23:00:01.220765 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:01.220725    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" event={"ID":"bbdb117f-e199-4424-9674-69c4f71224d4","Type":"ContainerStarted","Data":"fa4032a40a92252764dd691e6b7d624d2380f949d7e3edd577053d84037384b8"}
Nov 03 23:00:01.222115 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:01.222086    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" event={"ID":"f8cf17ac-f1dd-44dd-b3bb-627c61227327","Type":"ContainerStarted","Data":"508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487"}
Nov 03 23:00:01.223570 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:01.223541    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" event={"ID":"a876d4eb-4282-4ae9-9726-63a18708508f","Type":"ContainerStarted","Data":"b10139e03ac2243ca25583726f2066837ce98aacfd02208bcbf211604de6eba6"}
Nov 03 23:00:01.224905 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:01.224878    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" event={"ID":"9f7b63bc-125d-4b96-b401-c4dfcc396922","Type":"ContainerStarted","Data":"6572c5c7c55e7ec114e7abe1d19f5acc4fb30d193f84712b5d3a82f2be41e8e0"}
Nov 03 23:00:01.226097 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:01.226075    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2" event={"ID":"6bb8a3d7-b5fa-4761-98f3-065f9365d34a","Type":"ContainerStarted","Data":"a315ee0d3faba843ba544b68adabbfd3bc15efce471d1fc876c13c5cdaf6b7e4"}
Nov 03 23:00:02.234592 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.234560    5162 generic.go:356] "Generic (PLEG): container finished" podID="bbdb117f-e199-4424-9674-69c4f71224d4" containerID="827004bfc91a20c434d396aabe92811d393a0f65e7cd2bb5928a40d627e0623a" exitCode=60
Nov 03 23:00:02.235038 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.234644    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" event={"ID":"bbdb117f-e199-4424-9674-69c4f71224d4","Type":"ContainerDied","Data":"827004bfc91a20c434d396aabe92811d393a0f65e7cd2bb5928a40d627e0623a"}
Nov 03 23:00:02.235038 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.234942    5162 scope.go:122] "RemoveContainer" containerID="827004bfc91a20c434d396aabe92811d393a0f65e7cd2bb5928a40d627e0623a"
Nov 03 23:00:02.238236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.238211    5162 generic.go:356] "Generic (PLEG): container finished" podID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerID="b586999f4d1aba5964a397b723888e4940c76a928cabbfc5f3dd5dd98b4072a7" exitCode=7
Nov 03 23:00:02.238334 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.238301    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" event={"ID":"f8cf17ac-f1dd-44dd-b3bb-627c61227327","Type":"ContainerDied","Data":"b586999f4d1aba5964a397b723888e4940c76a928cabbfc5f3dd5dd98b4072a7"}
Nov 03 23:00:02.238623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.238557    5162 scope.go:122] "RemoveContainer" containerID="b586999f4d1aba5964a397b723888e4940c76a928cabbfc5f3dd5dd98b4072a7"
Nov 03 23:00:02.240129 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.240098    5162 generic.go:356] "Generic (PLEG): container finished" podID="a876d4eb-4282-4ae9-9726-63a18708508f" containerID="f6d8dc834e6a44aea19d910a5f6fa261af96d3a0b0241c68afb4bb6c6cc5f51f" exitCode=60
Nov 03 23:00:02.240209 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.240144    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" event={"ID":"a876d4eb-4282-4ae9-9726-63a18708508f","Type":"ContainerDied","Data":"f6d8dc834e6a44aea19d910a5f6fa261af96d3a0b0241c68afb4bb6c6cc5f51f"}
Nov 03 23:00:02.240528 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.240443    5162 scope.go:122] "RemoveContainer" containerID="f6d8dc834e6a44aea19d910a5f6fa261af96d3a0b0241c68afb4bb6c6cc5f51f"
Nov 03 23:00:02.242722 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.242705    5162 generic.go:356] "Generic (PLEG): container finished" podID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerID="0d866fec5dfc813cff92d931143b1e07a2e4c9b86b6df2ce171fa6c5c7e0c775" exitCode=60
Nov 03 23:00:02.242818 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.242760    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" event={"ID":"9f7b63bc-125d-4b96-b401-c4dfcc396922","Type":"ContainerDied","Data":"0d866fec5dfc813cff92d931143b1e07a2e4c9b86b6df2ce171fa6c5c7e0c775"}
Nov 03 23:00:02.242952 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.242938    5162 scope.go:122] "RemoveContainer" containerID="0d866fec5dfc813cff92d931143b1e07a2e4c9b86b6df2ce171fa6c5c7e0c775"
Nov 03 23:00:02.244420 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.244383    5162 generic.go:356] "Generic (PLEG): container finished" podID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerID="32f793419054a075e2f305c8c1ce2c264ca4eb86aad4ca6c70243d5f39ea7783" exitCode=60
Nov 03 23:00:02.244420 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.244416    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2" event={"ID":"6bb8a3d7-b5fa-4761-98f3-065f9365d34a","Type":"ContainerDied","Data":"32f793419054a075e2f305c8c1ce2c264ca4eb86aad4ca6c70243d5f39ea7783"}
Nov 03 23:00:02.244662 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:02.244647    5162 scope.go:122] "RemoveContainer" containerID="32f793419054a075e2f305c8c1ce2c264ca4eb86aad4ca6c70243d5f39ea7783"
Nov 03 23:00:03.255529 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.255506    5162 generic.go:356] "Generic (PLEG): container finished" podID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerID="ad6eade874eba1cd5ce41aaab66ff94c0fa9448df8139262baff0ed4196a1771" exitCode=60
Nov 03 23:00:03.255895 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.255595    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2" event={"ID":"6bb8a3d7-b5fa-4761-98f3-065f9365d34a","Type":"ContainerDied","Data":"ad6eade874eba1cd5ce41aaab66ff94c0fa9448df8139262baff0ed4196a1771"}
Nov 03 23:00:03.255895 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.255689    5162 scope.go:122] "RemoveContainer" containerID="32f793419054a075e2f305c8c1ce2c264ca4eb86aad4ca6c70243d5f39ea7783"
Nov 03 23:00:03.256098 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.256075    5162 scope.go:122] "RemoveContainer" containerID="ad6eade874eba1cd5ce41aaab66ff94c0fa9448df8139262baff0ed4196a1771"
Nov 03 23:00:03.256502 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:03.256477    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2_odh-ai-gateway-infra(6bb8a3d7-b5fa-4761-98f3-065f9365d34a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a"
Nov 03 23:00:03.258072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.258049    5162 generic.go:356] "Generic (PLEG): container finished" podID="bbdb117f-e199-4424-9674-69c4f71224d4" containerID="244fea7eec8371bab99dfece284e4e79d689453a3600d2eab2d7b8217269b050" exitCode=60
Nov 03 23:00:03.258175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.258129    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" event={"ID":"bbdb117f-e199-4424-9674-69c4f71224d4","Type":"ContainerDied","Data":"244fea7eec8371bab99dfece284e4e79d689453a3600d2eab2d7b8217269b050"}
Nov 03 23:00:03.258434 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.258418    5162 scope.go:122] "RemoveContainer" containerID="244fea7eec8371bab99dfece284e4e79d689453a3600d2eab2d7b8217269b050"
Nov 03 23:00:03.258746 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:03.258722    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h_odh-ai-gateway-infra(bbdb117f-e199-4424-9674-69c4f71224d4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" podUID="bbdb117f-e199-4424-9674-69c4f71224d4"
Nov 03 23:00:03.260208 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.260184    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" event={"ID":"f8cf17ac-f1dd-44dd-b3bb-627c61227327","Type":"ContainerStarted","Data":"78f97dc63d314bc130411439d3478d684b7c4311db37d826b8dbc62308f9b076"}
Nov 03 23:00:03.262179 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.262145    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" event={"ID":"a876d4eb-4282-4ae9-9726-63a18708508f","Type":"ContainerStarted","Data":"5d2ee5b5529a3f7404e70958f172b20fe5d1153602c744a57d6f3c7fe94b975b"}
Nov 03 23:00:03.263953 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.263933    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" event={"ID":"9f7b63bc-125d-4b96-b401-c4dfcc396922","Type":"ContainerStarted","Data":"fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13"}
Nov 03 23:00:03.264215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.264201    5162 scope.go:122] "RemoveContainer" containerID="fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13"
Nov 03 23:00:03.264431 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:03.264407    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677_odh-ai-gateway-infra(9f7b63bc-125d-4b96-b401-c4dfcc396922)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922"
Nov 03 23:00:03.480222 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.480196    5162 scope.go:122] "RemoveContainer" containerID="827004bfc91a20c434d396aabe92811d393a0f65e7cd2bb5928a40d627e0623a"
Nov 03 23:00:03.494934 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.494755    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" podStartSLOduration=3.494742106 podStartE2EDuration="3.494742106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:03.492377737 +0000 UTC m=+2206.757077747" watchObservedRunningTime="2026-11-03 23:00:03.494742106 +0000 UTC m=+2206.759442116"
Nov 03 23:00:03.851432 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:03.851312    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" podStartSLOduration=3.851297035 podStartE2EDuration="3.851297035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:03.847599695 +0000 UTC m=+2207.112299700" watchObservedRunningTime="2026-11-03 23:00:03.851297035 +0000 UTC m=+2207.115997071"
Nov 03 23:00:04.092809 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.092769    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:04.273950 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.273917    5162 scope.go:122] "RemoveContainer" containerID="244fea7eec8371bab99dfece284e4e79d689453a3600d2eab2d7b8217269b050"
Nov 03 23:00:04.274408 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:04.274305    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h_odh-ai-gateway-infra(bbdb117f-e199-4424-9674-69c4f71224d4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" podUID="bbdb117f-e199-4424-9674-69c4f71224d4"
Nov 03 23:00:04.275731 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.275704    5162 generic.go:356] "Generic (PLEG): container finished" podID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerID="78f97dc63d314bc130411439d3478d684b7c4311db37d826b8dbc62308f9b076" exitCode=7
Nov 03 23:00:04.275904 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.275802    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" event={"ID":"f8cf17ac-f1dd-44dd-b3bb-627c61227327","Type":"ContainerDied","Data":"78f97dc63d314bc130411439d3478d684b7c4311db37d826b8dbc62308f9b076"}
Nov 03 23:00:04.275904 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.275870    5162 scope.go:122] "RemoveContainer" containerID="b586999f4d1aba5964a397b723888e4940c76a928cabbfc5f3dd5dd98b4072a7"
Nov 03 23:00:04.276078 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.276046    5162 scope.go:122] "RemoveContainer" containerID="78f97dc63d314bc130411439d3478d684b7c4311db37d826b8dbc62308f9b076"
Nov 03 23:00:04.276415 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:04.276378    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp_odh-ai-gateway-infra(f8cf17ac-f1dd-44dd-b3bb-627c61227327)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327"
Nov 03 23:00:04.278210 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.278191    5162 generic.go:356] "Generic (PLEG): container finished" podID="a876d4eb-4282-4ae9-9726-63a18708508f" containerID="5d2ee5b5529a3f7404e70958f172b20fe5d1153602c744a57d6f3c7fe94b975b" exitCode=60
Nov 03 23:00:04.278308 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.278245    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" event={"ID":"a876d4eb-4282-4ae9-9726-63a18708508f","Type":"ContainerDied","Data":"5d2ee5b5529a3f7404e70958f172b20fe5d1153602c744a57d6f3c7fe94b975b"}
Nov 03 23:00:04.278563 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.278547    5162 scope.go:122] "RemoveContainer" containerID="5d2ee5b5529a3f7404e70958f172b20fe5d1153602c744a57d6f3c7fe94b975b"
Nov 03 23:00:04.278849 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:04.278830    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9_odh-ai-gateway-infra(a876d4eb-4282-4ae9-9726-63a18708508f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" podUID="a876d4eb-4282-4ae9-9726-63a18708508f"
Nov 03 23:00:04.280077 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.280056    5162 generic.go:356] "Generic (PLEG): container finished" podID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerID="fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13" exitCode=60
Nov 03 23:00:04.280154 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.280139    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" event={"ID":"9f7b63bc-125d-4b96-b401-c4dfcc396922","Type":"ContainerDied","Data":"fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13"}
Nov 03 23:00:04.280345 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.280331    5162 scope.go:122] "RemoveContainer" containerID="fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13"
Nov 03 23:00:04.280619 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:04.280600    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677_odh-ai-gateway-infra(9f7b63bc-125d-4b96-b401-c4dfcc396922)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922"
Nov 03 23:00:04.281793 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.281778    5162 scope.go:122] "RemoveContainer" containerID="ad6eade874eba1cd5ce41aaab66ff94c0fa9448df8139262baff0ed4196a1771"
Nov 03 23:00:04.282017 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:04.281999    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2_odh-ai-gateway-infra(6bb8a3d7-b5fa-4761-98f3-065f9365d34a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a"
Nov 03 23:00:04.290584 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.290568    5162 scope.go:122] "RemoveContainer" containerID="f6d8dc834e6a44aea19d910a5f6fa261af96d3a0b0241c68afb4bb6c6cc5f51f"
Nov 03 23:00:04.303605 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.303584    5162 scope.go:122] "RemoveContainer" containerID="0d866fec5dfc813cff92d931143b1e07a2e4c9b86b6df2ce171fa6c5c7e0c775"
Nov 03 23:00:04.624631 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.624542    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:04.634508 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:04.634446    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"]
Nov 03 23:00:05.291751 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.291719    5162 scope.go:122] "RemoveContainer" containerID="78f97dc63d314bc130411439d3478d684b7c4311db37d826b8dbc62308f9b076"
Nov 03 23:00:05.292174 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:05.292071    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp_odh-ai-gateway-infra(f8cf17ac-f1dd-44dd-b3bb-627c61227327)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327"
Nov 03 23:00:05.293377 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.293354    5162 scope.go:122] "RemoveContainer" containerID="5d2ee5b5529a3f7404e70958f172b20fe5d1153602c744a57d6f3c7fe94b975b"
Nov 03 23:00:05.293693 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:05.293667    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9_odh-ai-gateway-infra(a876d4eb-4282-4ae9-9726-63a18708508f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" podUID="a876d4eb-4282-4ae9-9726-63a18708508f"
Nov 03 23:00:05.295031 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.295011    5162 scope.go:122] "RemoveContainer" containerID="fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13"
Nov 03 23:00:05.295504 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:05.295257    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677_odh-ai-gateway-infra(9f7b63bc-125d-4b96-b401-c4dfcc396922)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922"
Nov 03 23:00:05.387843 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.387808    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:05.451995 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.451968    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:05.536442 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.530986    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-maas-api-service-ca\") pod \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") "
Nov 03 23:00:05.536442 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.531233    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-kube-api-access-tcsss\" (UniqueName: \"kubernetes.io/projected/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-kube-api-access-tcsss\") pod \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\" (UID: \"6bb8a3d7-b5fa-4761-98f3-065f9365d34a\") "
Nov 03 23:00:05.552464 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.546258    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-maas-api-service-ca" pod "6bb8a3d7-b5fa-4761-98f3-065f9365d34a" (UID: "6bb8a3d7-b5fa-4761-98f3-065f9365d34a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:05.579740 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.579696    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-kube-api-access-tcsss" pod "6bb8a3d7-b5fa-4761-98f3-065f9365d34a" (UID: "6bb8a3d7-b5fa-4761-98f3-065f9365d34a"). InnerVolumeSpecName "kube-api-access-tcsss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:05.594629 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.594551    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:05.632899 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.632862    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcsss\" (UniqueName: \"kubernetes.io/projected/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-kube-api-access-tcsss\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:05.632899 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.632897    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bb8a3d7-b5fa-4761-98f3-065f9365d34a-maas-api-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:05.885376 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:05.885292    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:06.203585 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.203414    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:06.304265 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.304216    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2" event={"ID":"6bb8a3d7-b5fa-4761-98f3-065f9365d34a","Type":"ContainerDied","Data":"a315ee0d3faba843ba544b68adabbfd3bc15efce471d1fc876c13c5cdaf6b7e4"}
Nov 03 23:00:06.304778 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.304288    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"
Nov 03 23:00:06.304778 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.304332    5162 scope.go:122] "RemoveContainer" containerID="ad6eade874eba1cd5ce41aaab66ff94c0fa9448df8139262baff0ed4196a1771"
Nov 03 23:00:06.494520 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.494450    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"]
Nov 03 23:00:06.500810 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.500783    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-2769cb-29895780-t8sj2"]
Nov 03 23:00:06.553994 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.553967    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:06.768543 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.762640    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:06.862838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:06.862804    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" path="/var/lib/kubelet/pods/6bb8a3d7-b5fa-4761-98f3-065f9365d34a/volumes"
Nov 03 23:00:07.248412 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:07.244554    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:07.783490 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:07.783443    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:08.200645 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:08.200602    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:08.535724 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:08.535630    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:09.183031 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:09.182991    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:09.646362 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:09.646267    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:10.085077 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:10.085041    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:00:11.150955 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:11.150916    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:11.417262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:11.417168    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"]
Nov 03 23:00:11.417474 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:11.417415    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy" containerID="cri-o://3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147" gracePeriod=30
Nov 03 23:00:11.419055 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:11.419025    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:11.434451 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:11.434428    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"]
Nov 03 23:00:11.434638 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:11.434620    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy" containerID="cri-o://dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3" gracePeriod=30
Nov 03 23:00:11.626416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:11.626361    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:12.995999 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:12.995969    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:13.534576 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:13.534538    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:13.741475 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:13.741439    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:13.839817 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:13.839748    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"]
Nov 03 23:00:13.840049 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:13.840021    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy" containerID="cri-o://56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3" gracePeriod=30
Nov 03 23:00:13.848040 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:13.848021    5162 scope.go:122] "RemoveContainer" containerID="78f97dc63d314bc130411439d3478d684b7c4311db37d826b8dbc62308f9b076"
Nov 03 23:00:13.958412 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:13.952331    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:14.411857 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:14.411772    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" event={"ID":"f8cf17ac-f1dd-44dd-b3bb-627c61227327","Type":"ContainerStarted","Data":"6d942cc0c65ed56c6cdc6814b53b70050e85aa01a59134c3521fce0b961e082b"}
Nov 03 23:00:15.421091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:15.421060    5162 generic.go:356] "Generic (PLEG): container finished" podID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerID="6d942cc0c65ed56c6cdc6814b53b70050e85aa01a59134c3521fce0b961e082b" exitCode=60
Nov 03 23:00:15.421537 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:15.421144    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" event={"ID":"f8cf17ac-f1dd-44dd-b3bb-627c61227327","Type":"ContainerDied","Data":"6d942cc0c65ed56c6cdc6814b53b70050e85aa01a59134c3521fce0b961e082b"}
Nov 03 23:00:15.421537 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:15.421207    5162 scope.go:122] "RemoveContainer" containerID="78f97dc63d314bc130411439d3478d684b7c4311db37d826b8dbc62308f9b076"
Nov 03 23:00:15.421630 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:15.421616    5162 scope.go:122] "RemoveContainer" containerID="6d942cc0c65ed56c6cdc6814b53b70050e85aa01a59134c3521fce0b961e082b"
Nov 03 23:00:15.421958 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:15.421941    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp_odh-ai-gateway-infra(f8cf17ac-f1dd-44dd-b3bb-627c61227327)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327"
Nov 03 23:00:15.541673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:15.541640    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"]
Nov 03 23:00:15.848063 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:15.847987    5162 scope.go:122] "RemoveContainer" containerID="244fea7eec8371bab99dfece284e4e79d689453a3600d2eab2d7b8217269b050"
Nov 03 23:00:15.863758 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:15.863728    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:16.060893 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.060858    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:16.274809 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.274072    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:16.430262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.430185    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" event={"ID":"bbdb117f-e199-4424-9674-69c4f71224d4","Type":"ContainerStarted","Data":"ff26666b80add7b2e874f9c9c5849a905431c90bb64cc694af4dfda271076c15"}
Nov 03 23:00:16.551456 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.551368    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" podStartSLOduration=16.55134812 podStartE2EDuration="16.55134812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:16.549663077 +0000 UTC m=+2219.814363089" watchObservedRunningTime="2026-11-03 23:00:16.55134812 +0000 UTC m=+2219.816048132"
Nov 03 23:00:16.608804 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.608777    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:16.688569 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.688538    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8cf17ac-f1dd-44dd-b3bb-627c61227327-kube-api-access-rfk7x\" (UniqueName: \"kubernetes.io/projected/f8cf17ac-f1dd-44dd-b3bb-627c61227327-kube-api-access-rfk7x\") pod \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") "
Nov 03 23:00:16.688789 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.688767    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8cf17ac-f1dd-44dd-b3bb-627c61227327-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cf17ac-f1dd-44dd-b3bb-627c61227327-maas-api-service-ca\") pod \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\" (UID: \"f8cf17ac-f1dd-44dd-b3bb-627c61227327\") "
Nov 03 23:00:16.689814 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.689787    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8cf17ac-f1dd-44dd-b3bb-627c61227327-maas-api-service-ca" pod "f8cf17ac-f1dd-44dd-b3bb-627c61227327" (UID: "f8cf17ac-f1dd-44dd-b3bb-627c61227327"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:16.692547 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.692518    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8cf17ac-f1dd-44dd-b3bb-627c61227327-kube-api-access-rfk7x" pod "f8cf17ac-f1dd-44dd-b3bb-627c61227327" (UID: "f8cf17ac-f1dd-44dd-b3bb-627c61227327"). InnerVolumeSpecName "kube-api-access-rfk7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:16.705696 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.705552    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 23:00:16.753907 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.753881    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 23:00:16.789975 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.789945    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-token\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-token\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790160 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790017    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-podinfo\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790252 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790221    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-socket\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790317 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790259    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istiod-ca-cert\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790372 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790343    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-envoy\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790439 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790426    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-data\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790494    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-kube-api-access-z4pnz\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-kube-api-access-z4pnz\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790503    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-socket" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.790632 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790547    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-certs\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790632 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790580    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-credential-socket\") pod \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\" (UID: \"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06\") "
Nov 03 23:00:16.790790 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790765    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istiod-ca-cert" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:16.791285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790994    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-data" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.791285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.790998    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-credential-socket" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.791285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791113    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-certs" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.791530 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791477    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.791530 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791501    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.791530 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791518    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8cf17ac-f1dd-44dd-b3bb-627c61227327-maas-api-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.791705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791532    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.791705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791545    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.791705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791558    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.791705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.791572    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfk7x\" (UniqueName: \"kubernetes.io/projected/f8cf17ac-f1dd-44dd-b3bb-627c61227327-kube-api-access-rfk7x\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.793020 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.792994    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-podinfo" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:00:16.793109 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.793086    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-kube-api-access-z4pnz" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "kube-api-access-z4pnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:16.793496 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.793478    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-envoy" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.793959 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.793940    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-token" pod "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" (UID: "daea8b8c-88b2-41d7-a1cf-7b4fd4306e06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:16.848057 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.848030    5162 scope.go:122] "RemoveContainer" containerID="fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13"
Nov 03 23:00:16.892309 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892282    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-socket\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892488 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892401    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-certs\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892488 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892438    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-data\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892488 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892474    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-kube-api-access-j57j4\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-kube-api-access-j57j4\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892687 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892525    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-credential-socket\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892687 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892551    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3eca86f5-f01a-4000-8118-27836d87ba36-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eca86f5-f01a-4000-8118-27836d87ba36-istiod-ca-cert\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892687 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892604    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3eca86f5-f01a-4000-8118-27836d87ba36-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eca86f5-f01a-4000-8118-27836d87ba36-istio-podinfo\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892839 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892750    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-envoy\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.892935 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892914    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-istio-token\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-istio-token\") pod \"3eca86f5-f01a-4000-8118-27836d87ba36\" (UID: \"3eca86f5-f01a-4000-8118-27836d87ba36\") "
Nov 03 23:00:16.893072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892601    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-socket" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.893138 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892612    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-certs" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.893138 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892822    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-data" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.893138 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.892930    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eca86f5-f01a-4000-8118-27836d87ba36-istiod-ca-cert" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:16.893307 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893143    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-credential-socket" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893636    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893662    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893681    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4pnz\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-kube-api-access-z4pnz\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893696    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893709    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893723    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893738    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eca86f5-f01a-4000-8118-27836d87ba36-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893752    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.893788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.893767    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.895370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.895334    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-kube-api-access-j57j4" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "kube-api-access-j57j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:16.895471 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.895383    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eca86f5-f01a-4000-8118-27836d87ba36-istio-podinfo" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:00:16.895548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.895527    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-istio-token" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:16.895659 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.895642    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-envoy" pod "3eca86f5-f01a-4000-8118-27836d87ba36" (UID: "3eca86f5-f01a-4000-8118-27836d87ba36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:16.995442 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.995377    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eca86f5-f01a-4000-8118-27836d87ba36-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.995672 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.995636    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eca86f5-f01a-4000-8118-27836d87ba36-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.995672 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.995664    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:16.995884 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:16.995681    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j57j4\" (UniqueName: \"kubernetes.io/projected/3eca86f5-f01a-4000-8118-27836d87ba36-kube-api-access-j57j4\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:17.443207 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.443165    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" event={"ID":"9f7b63bc-125d-4b96-b401-c4dfcc396922","Type":"ContainerStarted","Data":"c5165591c0a4e434e8ce728d893b391b54bb74b73ce50b150007b0ca6018ff9b"}
Nov 03 23:00:17.446330 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.446304    5162 generic.go:356] "Generic (PLEG): container finished" podID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerID="3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147" exitCode=0
Nov 03 23:00:17.446503 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.446473    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" event={"ID":"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06","Type":"ContainerDied","Data":"3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147"}
Nov 03 23:00:17.446580 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.446522    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69" event={"ID":"daea8b8c-88b2-41d7-a1cf-7b4fd4306e06","Type":"ContainerDied","Data":"339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d"}
Nov 03 23:00:17.446580 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.446552    5162 scope.go:122] "RemoveContainer" containerID="3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147"
Nov 03 23:00:17.446706 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.446695    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"
Nov 03 23:00:17.450302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.450277    5162 generic.go:356] "Generic (PLEG): container finished" podID="bbdb117f-e199-4424-9674-69c4f71224d4" containerID="ff26666b80add7b2e874f9c9c5849a905431c90bb64cc694af4dfda271076c15" exitCode=60
Nov 03 23:00:17.450478 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.450421    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" event={"ID":"bbdb117f-e199-4424-9674-69c4f71224d4","Type":"ContainerDied","Data":"ff26666b80add7b2e874f9c9c5849a905431c90bb64cc694af4dfda271076c15"}
Nov 03 23:00:17.450851 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.450830    5162 scope.go:122] "RemoveContainer" containerID="ff26666b80add7b2e874f9c9c5849a905431c90bb64cc694af4dfda271076c15"
Nov 03 23:00:17.451185 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:17.451161    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h_odh-ai-gateway-infra(bbdb117f-e199-4424-9674-69c4f71224d4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" podUID="bbdb117f-e199-4424-9674-69c4f71224d4"
Nov 03 23:00:17.453069 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.453047    5162 generic.go:356] "Generic (PLEG): container finished" podID="3eca86f5-f01a-4000-8118-27836d87ba36" containerID="dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3" exitCode=0
Nov 03 23:00:17.453194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.453124    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"
Nov 03 23:00:17.453194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.453149    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" event={"ID":"3eca86f5-f01a-4000-8118-27836d87ba36","Type":"ContainerDied","Data":"dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3"}
Nov 03 23:00:17.453194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.453184    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758" event={"ID":"3eca86f5-f01a-4000-8118-27836d87ba36","Type":"ContainerDied","Data":"ee639b71ebbf5031afed161530804cf4bb8f1c87185beacf0e2f57aac944dcff"}
Nov 03 23:00:17.455208 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.455065    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp" event={"ID":"f8cf17ac-f1dd-44dd-b3bb-627c61227327","Type":"ContainerDied","Data":"508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487"}
Nov 03 23:00:17.455897 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.455351    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"
Nov 03 23:00:17.461324 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.461306    5162 scope.go:122] "RemoveContainer" containerID="3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147"
Nov 03 23:00:17.461625 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:17.461604    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147\": container with ID starting with 3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147 not found: ID does not exist" containerID="3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147"
Nov 03 23:00:17.461716 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.461637    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147"} err="failed to get container status \"3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147\": rpc error: code = NotFound desc = could not find container \"3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147\": container with ID starting with 3c884418da47426dc6bae06416ea751f546969ab8a179956dc4dfa74ad984147 not found: ID does not exist"
Nov 03 23:00:17.461716 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.461661    5162 scope.go:122] "RemoveContainer" containerID="244fea7eec8371bab99dfece284e4e79d689453a3600d2eab2d7b8217269b050"
Nov 03 23:00:17.482790 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.482763    5162 scope.go:122] "RemoveContainer" containerID="dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3"
Nov 03 23:00:17.493281 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.493261    5162 scope.go:122] "RemoveContainer" containerID="dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3"
Nov 03 23:00:17.493610 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:17.493587    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3\": container with ID starting with dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3 not found: ID does not exist" containerID="dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3"
Nov 03 23:00:17.493678 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.493619    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3"} err="failed to get container status \"dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3\": rpc error: code = NotFound desc = could not find container \"dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3\": container with ID starting with dd9735567820026349ee0dd5b3eb0367b04c69521ce9c1c241ec20742ffd66c3 not found: ID does not exist"
Nov 03 23:00:17.493678 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.493638    5162 scope.go:122] "RemoveContainer" containerID="6d942cc0c65ed56c6cdc6814b53b70050e85aa01a59134c3521fce0b961e082b"
Nov 03 23:00:17.494834 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:17.494813    5162 kuberuntime_gc.go:392] "Failed to remove container log dead symlink" err="remove /var/log/containers/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp_odh-ai-gateway-infra_cleanup-6d942cc0c65ed56c6cdc6814b53b70050e85aa01a59134c3521fce0b961e082b.log: no such file or directory" path="/var/log/containers/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp_odh-ai-gateway-infra_cleanup-6d942cc0c65ed56c6cdc6814b53b70050e85aa01a59134c3521fce0b961e082b.log"
Nov 03 23:00:17.563575 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.563545    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"]
Nov 03 23:00:17.566941 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.566884    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" podStartSLOduration=17.56686783 podStartE2EDuration="17.56686783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:17.562156533 +0000 UTC m=+2220.826856544" watchObservedRunningTime="2026-11-03 23:00:17.56686783 +0000 UTC m=+2220.831567843"
Nov 03 23:00:17.679840 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.679796    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"]
Nov 03 23:00:17.684642 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.684617    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69"]
Nov 03 23:00:17.801507 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.801470    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"]
Nov 03 23:00:17.806971 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.806940    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758"]
Nov 03 23:00:17.847084 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.847058    5162 scope.go:122] "RemoveContainer" containerID="5d2ee5b5529a3f7404e70958f172b20fe5d1153602c744a57d6f3c7fe94b975b"
Nov 03 23:00:17.917455 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.917422    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"]
Nov 03 23:00:17.923775 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:17.923751    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-31ad7b-29895780-fwqbp"]
Nov 03 23:00:18.255016 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.254979    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:18.473419 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.472140    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" event={"ID":"a876d4eb-4282-4ae9-9726-63a18708508f","Type":"ContainerStarted","Data":"b636958b73c9f7025f3f225f42672e6fef481dfa166b2ee6800172bbaafeb7ab"}
Nov 03 23:00:18.473419 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.472356    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:18.474362 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.474335    5162 generic.go:356] "Generic (PLEG): container finished" podID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerID="c5165591c0a4e434e8ce728d893b391b54bb74b73ce50b150007b0ca6018ff9b" exitCode=60
Nov 03 23:00:18.474518 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.474421    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" event={"ID":"9f7b63bc-125d-4b96-b401-c4dfcc396922","Type":"ContainerDied","Data":"c5165591c0a4e434e8ce728d893b391b54bb74b73ce50b150007b0ca6018ff9b"}
Nov 03 23:00:18.474518 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.474480    5162 scope.go:122] "RemoveContainer" containerID="fdbc766b779fd46fede1c4cb8730d5154c0f6d39446cd99e89142af64b091b13"
Nov 03 23:00:18.475379 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.474829    5162 scope.go:122] "RemoveContainer" containerID="c5165591c0a4e434e8ce728d893b391b54bb74b73ce50b150007b0ca6018ff9b"
Nov 03 23:00:18.475379 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:18.475164    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677_odh-ai-gateway-infra(9f7b63bc-125d-4b96-b401-c4dfcc396922)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922"
Nov 03 23:00:18.573781 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.573749    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"]
Nov 03 23:00:18.580338 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:18.580301    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:18.631615 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.631591    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:18.718983 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.718906    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbdb117f-e199-4424-9674-69c4f71224d4-kube-api-access-sjm78\" (UniqueName: \"kubernetes.io/projected/bbdb117f-e199-4424-9674-69c4f71224d4-kube-api-access-sjm78\") pod \"bbdb117f-e199-4424-9674-69c4f71224d4\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") "
Nov 03 23:00:18.719194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.719034    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbdb117f-e199-4424-9674-69c4f71224d4-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdb117f-e199-4424-9674-69c4f71224d4-maas-api-service-ca\") pod \"bbdb117f-e199-4424-9674-69c4f71224d4\" (UID: \"bbdb117f-e199-4424-9674-69c4f71224d4\") "
Nov 03 23:00:18.719404 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.719369    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbdb117f-e199-4424-9674-69c4f71224d4-maas-api-service-ca" pod "bbdb117f-e199-4424-9674-69c4f71224d4" (UID: "bbdb117f-e199-4424-9674-69c4f71224d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:18.719531 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.719517    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdb117f-e199-4424-9674-69c4f71224d4-maas-api-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:18.720847 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.720825    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbdb117f-e199-4424-9674-69c4f71224d4-kube-api-access-sjm78" pod "bbdb117f-e199-4424-9674-69c4f71224d4" (UID: "bbdb117f-e199-4424-9674-69c4f71224d4"). InnerVolumeSpecName "kube-api-access-sjm78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:18.820915 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.820881    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjm78\" (UniqueName: \"kubernetes.io/projected/bbdb117f-e199-4424-9674-69c4f71224d4-kube-api-access-sjm78\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:18.858636 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.858610    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" path="/var/lib/kubelet/pods/3eca86f5-f01a-4000-8118-27836d87ba36/volumes"
Nov 03 23:00:18.859129 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.859115    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" path="/var/lib/kubelet/pods/daea8b8c-88b2-41d7-a1cf-7b4fd4306e06/volumes"
Nov 03 23:00:18.859522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:18.859504    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" path="/var/lib/kubelet/pods/f8cf17ac-f1dd-44dd-b3bb-627c61227327/volumes"
Nov 03 23:00:19.145050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.145024    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 23:00:19.327861 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.327779    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-envoy\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328023 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.327900    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-token\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-token\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328023 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.327941    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-podinfo\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328023 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.327973    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-credential-socket\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328077    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-kube-api-access-8bd9n\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-kube-api-access-8bd9n\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328241 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328174    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-socket\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328303 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328286    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-certs\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328357 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328313    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-data\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328428 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328376    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istiod-ca-cert\") pod \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\" (UID: \"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051\") "
Nov 03 23:00:19.328490 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328462    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-socket" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:19.328602 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328575    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-certs" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:19.328774 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328734    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-data" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:19.328888 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328869    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-credential-socket" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:19.328972 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.328938    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istiod-ca-cert" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:19.329111 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.329069    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.329111 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.329086    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.329226 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.329115    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.330623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.330595    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-envoy" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:19.330623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.330616    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-token" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:19.330784 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.330601    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-kube-api-access-8bd9n" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "kube-api-access-8bd9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:19.331039 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.331015    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-podinfo" pod "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" (UID: "9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:00:19.431031 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.430998    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.431031 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.431030    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.431251 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.431043    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.431251 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.431055    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.431251 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.431068    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.431251 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.431081    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bd9n\" (UniqueName: \"kubernetes.io/projected/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051-kube-api-access-8bd9n\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.484871 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.484846    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"
Nov 03 23:00:19.485502 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.484909    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h" event={"ID":"bbdb117f-e199-4424-9674-69c4f71224d4","Type":"ContainerDied","Data":"fa4032a40a92252764dd691e6b7d624d2380f949d7e3edd577053d84037384b8"}
Nov 03 23:00:19.485502 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.485011    5162 scope.go:122] "RemoveContainer" containerID="ff26666b80add7b2e874f9c9c5849a905431c90bb64cc694af4dfda271076c15"
Nov 03 23:00:19.488279 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.488168    5162 generic.go:356] "Generic (PLEG): container finished" podID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerID="56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3" exitCode=0
Nov 03 23:00:19.488422 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.488349    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"
Nov 03 23:00:19.490664 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.490637    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" event={"ID":"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051","Type":"ContainerDied","Data":"56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3"}
Nov 03 23:00:19.491012 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.490698    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q" event={"ID":"9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051","Type":"ContainerDied","Data":"63eebde652dc0ffd33b16c8df4b2d72f489e11e1217056ca98ad4f5edbb21845"}
Nov 03 23:00:19.495305 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.495283    5162 generic.go:356] "Generic (PLEG): container finished" podID="a876d4eb-4282-4ae9-9726-63a18708508f" containerID="b636958b73c9f7025f3f225f42672e6fef481dfa166b2ee6800172bbaafeb7ab" exitCode=60
Nov 03 23:00:19.495432 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.495328    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" event={"ID":"a876d4eb-4282-4ae9-9726-63a18708508f","Type":"ContainerDied","Data":"b636958b73c9f7025f3f225f42672e6fef481dfa166b2ee6800172bbaafeb7ab"}
Nov 03 23:00:19.495755 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.495738    5162 scope.go:122] "RemoveContainer" containerID="b636958b73c9f7025f3f225f42672e6fef481dfa166b2ee6800172bbaafeb7ab"
Nov 03 23:00:19.496088 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:19.496066    5162 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9_odh-ai-gateway-infra(a876d4eb-4282-4ae9-9726-63a18708508f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" podUID="a876d4eb-4282-4ae9-9726-63a18708508f"
Nov 03 23:00:19.498973 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.498954    5162 scope.go:122] "RemoveContainer" containerID="56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3"
Nov 03 23:00:19.509572 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.509552    5162 scope.go:122] "RemoveContainer" containerID="1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908"
Nov 03 23:00:19.521517 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.521428    5162 scope.go:122] "RemoveContainer" containerID="56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3"
Nov 03 23:00:19.521840 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:19.521810    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3\": container with ID starting with 56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3 not found: ID does not exist" containerID="56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3"
Nov 03 23:00:19.521897 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.521852    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3"} err="failed to get container status \"56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3\": rpc error: code = NotFound desc = could not find container \"56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3\": container with ID starting with 56a0aeb86bf0c17577bfd72dcdb3f207b50e8c27183cf5ba27301ba932b9c2b3 not found: ID does not exist"
Nov 03 23:00:19.521897 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.521871    5162 scope.go:122] "RemoveContainer" containerID="1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908"
Nov 03 23:00:19.522171 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:19.522151    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908\": container with ID starting with 1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908 not found: ID does not exist" containerID="1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908"
Nov 03 23:00:19.522263 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.522179    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908"} err="failed to get container status \"1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908\": rpc error: code = NotFound desc = could not find container \"1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908\": container with ID starting with 1e63e17d0116c6c456c632b1ad5213f41af9aaa642aaba2e51d562cc9d99d908 not found: ID does not exist"
Nov 03 23:00:19.522263 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.522203    5162 scope.go:122] "RemoveContainer" containerID="5d2ee5b5529a3f7404e70958f172b20fe5d1153602c744a57d6f3c7fe94b975b"
Nov 03 23:00:19.626190 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.626162    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:19.719525 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.719490    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"]
Nov 03 23:00:19.724529 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.724502    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"]
Nov 03 23:00:19.735116 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.735093    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f7b63bc-125d-4b96-b401-c4dfcc396922-kube-api-access-sb7vm\" (UniqueName: \"kubernetes.io/projected/9f7b63bc-125d-4b96-b401-c4dfcc396922-kube-api-access-sb7vm\") pod \"9f7b63bc-125d-4b96-b401-c4dfcc396922\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") "
Nov 03 23:00:19.735269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.735203    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f7b63bc-125d-4b96-b401-c4dfcc396922-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7b63bc-125d-4b96-b401-c4dfcc396922-maas-api-service-ca\") pod \"9f7b63bc-125d-4b96-b401-c4dfcc396922\" (UID: \"9f7b63bc-125d-4b96-b401-c4dfcc396922\") "
Nov 03 23:00:19.735601 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.735576    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f7b63bc-125d-4b96-b401-c4dfcc396922-maas-api-service-ca" pod "9f7b63bc-125d-4b96-b401-c4dfcc396922" (UID: "9f7b63bc-125d-4b96-b401-c4dfcc396922"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:19.737280 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.737257    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f7b63bc-125d-4b96-b401-c4dfcc396922-kube-api-access-sb7vm" pod "9f7b63bc-125d-4b96-b401-c4dfcc396922" (UID: "9f7b63bc-125d-4b96-b401-c4dfcc396922"). InnerVolumeSpecName "kube-api-access-sb7vm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:19.771138 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.771113    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q"]
Nov 03 23:00:19.837132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.837104    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb7vm\" (UniqueName: \"kubernetes.io/projected/9f7b63bc-125d-4b96-b401-c4dfcc396922-kube-api-access-sb7vm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.837132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.837129    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7b63bc-125d-4b96-b401-c4dfcc396922-maas-api-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.980436 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.980401    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"]
Nov 03 23:00:19.988259 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:19.988227    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-d6296b-29895780-6546h"]
Nov 03 23:00:20.045183 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.045147    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:20.505680 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.505654    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"
Nov 03 23:00:20.506065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.505683    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677" event={"ID":"9f7b63bc-125d-4b96-b401-c4dfcc396922","Type":"ContainerDied","Data":"6572c5c7c55e7ec114e7abe1d19f5acc4fb30d193f84712b5d3a82f2be41e8e0"}
Nov 03 23:00:20.506065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.505745    5162 scope.go:122] "RemoveContainer" containerID="c5165591c0a4e434e8ce728d893b391b54bb74b73ce50b150007b0ca6018ff9b"
Nov 03 23:00:20.668700 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.668672    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:20.741939 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.741903    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"]
Nov 03 23:00:20.746772 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.746741    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-68fbd1-29895780-wn677"]
Nov 03 23:00:20.850846 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.850768    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a876d4eb-4282-4ae9-9726-63a18708508f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a876d4eb-4282-4ae9-9726-63a18708508f-maas-api-service-ca\") pod \"a876d4eb-4282-4ae9-9726-63a18708508f\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") "
Nov 03 23:00:20.851026 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.850911    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a876d4eb-4282-4ae9-9726-63a18708508f-kube-api-access-d4xb6\" (UniqueName: \"kubernetes.io/projected/a876d4eb-4282-4ae9-9726-63a18708508f-kube-api-access-d4xb6\") pod \"a876d4eb-4282-4ae9-9726-63a18708508f\" (UID: \"a876d4eb-4282-4ae9-9726-63a18708508f\") "
Nov 03 23:00:20.851183 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.851158    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a876d4eb-4282-4ae9-9726-63a18708508f-maas-api-service-ca" pod "a876d4eb-4282-4ae9-9726-63a18708508f" (UID: "a876d4eb-4282-4ae9-9726-63a18708508f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:20.851578 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.851554    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a876d4eb-4282-4ae9-9726-63a18708508f-maas-api-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:20.853180 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.853155    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a876d4eb-4282-4ae9-9726-63a18708508f-kube-api-access-d4xb6" pod "a876d4eb-4282-4ae9-9726-63a18708508f" (UID: "a876d4eb-4282-4ae9-9726-63a18708508f"). InnerVolumeSpecName "kube-api-access-d4xb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:20.858071 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.858047    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" path="/var/lib/kubelet/pods/9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051/volumes"
Nov 03 23:00:20.858525 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.858510    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" path="/var/lib/kubelet/pods/9f7b63bc-125d-4b96-b401-c4dfcc396922/volumes"
Nov 03 23:00:20.858907 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.858894    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" path="/var/lib/kubelet/pods/bbdb117f-e199-4424-9674-69c4f71224d4/volumes"
Nov 03 23:00:20.953053 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:20.953020    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4xb6\" (UniqueName: \"kubernetes.io/projected/a876d4eb-4282-4ae9-9726-63a18708508f-kube-api-access-d4xb6\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:21.516961 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:21.516924    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"
Nov 03 23:00:21.517385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:21.516924    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9" event={"ID":"a876d4eb-4282-4ae9-9726-63a18708508f","Type":"ContainerDied","Data":"b10139e03ac2243ca25583726f2066837ce98aacfd02208bcbf211604de6eba6"}
Nov 03 23:00:21.517385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:21.517060    5162 scope.go:122] "RemoveContainer" containerID="b636958b73c9f7025f3f225f42672e6fef481dfa166b2ee6800172bbaafeb7ab"
Nov 03 23:00:21.687760 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:21.687727    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"]
Nov 03 23:00:21.694399 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:21.694365    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-5e72ad-29895780-p47c9"]
Nov 03 23:00:21.960795 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:21.960757    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:22.855064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:22.855028    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" path="/var/lib/kubelet/pods/a876d4eb-4282-4ae9-9726-63a18708508f/volumes"
Nov 03 23:00:23.711669 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:23.711625    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:24.511314 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:24.511283    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:26.886251 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:26.886214    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:27.097287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:27.097251    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:28.300581 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:28.300544    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:28.677160 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:28.677125    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:30.216021 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:30.215985    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:31.815791 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:31.815747    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"]
Nov 03 23:00:32.034896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:32.034854    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:32.127212 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:32.127122    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:32.127701 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:32.127659    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:35.446090 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:35.446053    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:35.643590 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:35.643557    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:37.030118 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:37.030072    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:38.691516 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:38.691471    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:38.873461 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:38.873429    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"]
Nov 03 23:00:39.207024 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:39.205649    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:39.414247 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:39.414182    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:40.085051 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:40.085018    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:00:40.385142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:40.385026    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:40.597637 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:40.597602    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:40.807823 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:40.807786    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:43.874542 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:43.874490    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" podUID="07b6e3a4-895b-4344-8897-22455346de35" containerName="payload-pre-processing" containerID="cri-o://38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84" gracePeriod=25
Nov 03 23:00:45.656822 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:45.656790    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:46.839632 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:46.839562    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" containerID="cri-o://464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0" gracePeriod=45
Nov 03 23:00:46.922313 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:46.922274    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": dial tcp 10.134.0.65:8000: connect: connection refused"
Nov 03 23:00:47.106662 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.106603    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 23:00:47.150777 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.150749    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-tmp-dir\") pod \"4783432e-dd9f-4416-b435-c6960145a39c\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") "
Nov 03 23:00:47.150928 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.150811    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-model-cache\") pod \"4783432e-dd9f-4416-b435-c6960145a39c\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") "
Nov 03 23:00:47.150928 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.150859    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4783432e-dd9f-4416-b435-c6960145a39c-kube-api-access-hgqf4\" (UniqueName: \"kubernetes.io/projected/4783432e-dd9f-4416-b435-c6960145a39c-kube-api-access-hgqf4\") pod \"4783432e-dd9f-4416-b435-c6960145a39c\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") "
Nov 03 23:00:47.151017 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.150974    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-dshm\") pod \"4783432e-dd9f-4416-b435-c6960145a39c\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") "
Nov 03 23:00:47.151064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151010    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-model-cache" pod "4783432e-dd9f-4416-b435-c6960145a39c" (UID: "4783432e-dd9f-4416-b435-c6960145a39c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:47.151064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151020    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-tmp-dir" pod "4783432e-dd9f-4416-b435-c6960145a39c" (UID: "4783432e-dd9f-4416-b435-c6960145a39c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:47.151145 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151113    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4783432e-dd9f-4416-b435-c6960145a39c-tls-certs\" (UniqueName: \"kubernetes.io/secret/4783432e-dd9f-4416-b435-c6960145a39c-tls-certs\") pod \"4783432e-dd9f-4416-b435-c6960145a39c\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") "
Nov 03 23:00:47.151225 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151211    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-home\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-home\") pod \"4783432e-dd9f-4416-b435-c6960145a39c\" (UID: \"4783432e-dd9f-4416-b435-c6960145a39c\") "
Nov 03 23:00:47.151525 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151504    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-home" pod "4783432e-dd9f-4416-b435-c6960145a39c" (UID: "4783432e-dd9f-4416-b435-c6960145a39c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:47.151790 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151774    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:47.151862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151790    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:47.151862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.151800    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:47.153190 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.153167    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4783432e-dd9f-4416-b435-c6960145a39c-tls-certs" pod "4783432e-dd9f-4416-b435-c6960145a39c" (UID: "4783432e-dd9f-4416-b435-c6960145a39c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:00:47.153190 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.153177    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-dshm" pod "4783432e-dd9f-4416-b435-c6960145a39c" (UID: "4783432e-dd9f-4416-b435-c6960145a39c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:47.153306 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.153188    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4783432e-dd9f-4416-b435-c6960145a39c-kube-api-access-hgqf4" pod "4783432e-dd9f-4416-b435-c6960145a39c" (UID: "4783432e-dd9f-4416-b435-c6960145a39c"). InnerVolumeSpecName "kube-api-access-hgqf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:47.253175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.253144    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4783432e-dd9f-4416-b435-c6960145a39c-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:47.253175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.253175    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgqf4\" (UniqueName: \"kubernetes.io/projected/4783432e-dd9f-4416-b435-c6960145a39c-kube-api-access-hgqf4\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:47.253175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.253185    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4783432e-dd9f-4416-b435-c6960145a39c-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:00:47.718766 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.718724    5162 generic.go:356] "Generic (PLEG): container finished" podID="4783432e-dd9f-4416-b435-c6960145a39c" containerID="464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0" exitCode=0
Nov 03 23:00:47.718921 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.718778    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"
Nov 03 23:00:47.718921 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.718814    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" event={"ID":"4783432e-dd9f-4416-b435-c6960145a39c","Type":"ContainerDied","Data":"464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0"}
Nov 03 23:00:47.718921 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.718860    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx" event={"ID":"4783432e-dd9f-4416-b435-c6960145a39c","Type":"ContainerDied","Data":"b925a6bc156fb27cb584cabc3b2fbb659db7af616862700d3faa66a56b5c357a"}
Nov 03 23:00:47.718921 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.718877    5162 scope.go:122] "RemoveContainer" containerID="464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0"
Nov 03 23:00:47.729574 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.729554    5162 scope.go:122] "RemoveContainer" containerID="464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0"
Nov 03 23:00:47.729833 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:47.729815    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0\": container with ID starting with 464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0 not found: ID does not exist" containerID="464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0"
Nov 03 23:00:47.729901 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.729841    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0"} err="failed to get container status \"464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0\": rpc error: code = NotFound desc = could not find container \"464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0\": container with ID starting with 464ab900a5f145038ab775641f157a386d0d2f80fc087096fb38fc4c9672d9b0 not found: ID does not exist"
Nov 03 23:00:47.888470 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.888433    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"]
Nov 03 23:00:47.939944 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:47.939917    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-961054/distinct-2-w3-961054-kserve-f95cb6bfd-2t6dx"]
Nov 03 23:00:48.764359 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:48.764324    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:48.854769 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:48.854738    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4783432e-dd9f-4416-b435-c6960145a39c" path="/var/lib/kubelet/pods/4783432e-dd9f-4416-b435-c6960145a39c/volumes"
Nov 03 23:00:50.130611 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:50.130578    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:51.967670 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:51.967623    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:53.030440 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:53.030385    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" podUID="07b6e3a4-895b-4344-8897-22455346de35" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 23:00:55.001427 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:55.001321    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:00:58.833505 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:00:58.833467    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache]"
Nov 03 23:00:59.235691 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:00:59.235655    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:00.774615 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:00.774582    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:01.023819 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:01.023781    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:01.235148 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:01.235113    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:01.684137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:01.684101    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:03.030025 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:03.029990    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" podUID="07b6e3a4-895b-4344-8897-22455346de35" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 23:01:04.179276 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.179249    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 23:01:04.302030 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.302000    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07b6e3a4-895b-4344-8897-22455346de35-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07b6e3a4-895b-4344-8897-22455346de35-plugins-config-volume\") pod \"07b6e3a4-895b-4344-8897-22455346de35\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") "
Nov 03 23:01:04.302232 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.302219    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07b6e3a4-895b-4344-8897-22455346de35-kube-api-access-k9wkt\" (UniqueName: \"kubernetes.io/projected/07b6e3a4-895b-4344-8897-22455346de35-kube-api-access-k9wkt\") pod \"07b6e3a4-895b-4344-8897-22455346de35\" (UID: \"07b6e3a4-895b-4344-8897-22455346de35\") "
Nov 03 23:01:04.302492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.302462    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b6e3a4-895b-4344-8897-22455346de35-plugins-config-volume" pod "07b6e3a4-895b-4344-8897-22455346de35" (UID: "07b6e3a4-895b-4344-8897-22455346de35"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:01:04.302913 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.302889    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07b6e3a4-895b-4344-8897-22455346de35-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:04.307572 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.307544    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b6e3a4-895b-4344-8897-22455346de35-kube-api-access-k9wkt" pod "07b6e3a4-895b-4344-8897-22455346de35" (UID: "07b6e3a4-895b-4344-8897-22455346de35"). InnerVolumeSpecName "kube-api-access-k9wkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:04.404228 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.404147    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9wkt\" (UniqueName: \"kubernetes.io/projected/07b6e3a4-895b-4344-8897-22455346de35-kube-api-access-k9wkt\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:04.832289 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.832253    5162 generic.go:356] "Generic (PLEG): container finished" podID="07b6e3a4-895b-4344-8897-22455346de35" containerID="38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84" exitCode=0
Nov 03 23:01:04.832497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.832315    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"
Nov 03 23:01:04.832497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.832329    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" event={"ID":"07b6e3a4-895b-4344-8897-22455346de35","Type":"ContainerDied","Data":"38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84"}
Nov 03 23:01:04.832497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.832428    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr" event={"ID":"07b6e3a4-895b-4344-8897-22455346de35","Type":"ContainerDied","Data":"27626ae9a58a256588d18d93d01d854ffc450080f0c7f92cc382f9052c84a205"}
Nov 03 23:01:04.832497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.832451    5162 scope.go:122] "RemoveContainer" containerID="38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84"
Nov 03 23:01:04.843252 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.843234    5162 scope.go:122] "RemoveContainer" containerID="38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84"
Nov 03 23:01:04.843546 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:01:04.843523    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84\": container with ID starting with 38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84 not found: ID does not exist" containerID="38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84"
Nov 03 23:01:04.843600 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:04.843554    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84"} err="failed to get container status \"38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84\": rpc error: code = NotFound desc = could not find container \"38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84\": container with ID starting with 38e4dad67f6dec5133b43e15de3da0149d15c5de448d58a11da6bfae4877da84 not found: ID does not exist"
Nov 03 23:01:05.004034 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:05.004004    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"]
Nov 03 23:01:05.055703 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:05.055663    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr"]
Nov 03 23:01:05.906127 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:05.906088    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:06.422451 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:06.422417    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:06.855499 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:06.855384    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07b6e3a4-895b-4344-8897-22455346de35" path="/var/lib/kubelet/pods/07b6e3a4-895b-4344-8897-22455346de35/volumes"
Nov 03 23:01:07.012985 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:01:07.012947    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache]"
Nov 03 23:01:07.792644 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:07.792553    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:08.084795 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:08.084702    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:08.305997 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:08.305963    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:08.636699 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:08.636664    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:08.848187 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:01:08.848144    5162 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cf17ac_f1dd_44dd_b3bb_627c61227327.slice/crio-508b9e2fb079cd84bc242223d6ea04cf1f1cefe447b72c7883e364c084c4a487\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice/crio-339014afa08c1a5e680dd793fa44dddde8346eeff76670741e71f1599958800d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaea8b8c_88b2_41d7_a1cf_7b4fd4306e06.slice\": RecentStats: unable to find data in memory cache]"
Nov 03 23:01:08.887143 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:08.887060    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:09.098818 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:09.098788    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:09.660973 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:09.660941    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:09.862713 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:09.862682    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:10.084353 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:01:10.084315    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:01:10.161834 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:10.161801    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:10.695274 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:10.695237    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:10.899345 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:10.899307    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:11.192950 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:11.192920    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:11.910138 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:11.910103    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:14.417540 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:14.417501    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:14.740200 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:14.740163    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:15.111882 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:15.111794    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:15.449212 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:15.449179    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:15.703150 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:15.703066    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:17.081050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:17.081017    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:17.293873 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:17.293829    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:18.180143 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:18.180110    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:18.446813 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:18.443065    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:20.267544 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:20.267509    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:20.590468 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:20.588532    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:23.341048 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:23.341010    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:23.631757 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:23.631673    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:23.835844 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:23.835808    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:24.180677 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:24.180628    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:24.381490 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:24.381453    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:30.322023 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:30.321985    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:30.606718 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:30.606631    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:30.814049 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:30.814008    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:31.245834 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:31.245793    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:35.980898 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:35.980862    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:37.343962 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:37.343926    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:37.600687 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:37.600599    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:37.626644 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:37.626615    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"]
Nov 03 23:01:37.626840 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:37.626820    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy" containerID="cri-o://d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef" gracePeriod=30
Nov 03 23:01:37.816338 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:37.816302    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:40.085809 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:01:40.085766    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:01:41.150082 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:41.150006    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:41.514639 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:41.514603    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:42.903021 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.902959    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 23:01:42.925483 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925459    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-socket\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925598 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925499    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-kube-api-access-7xk57\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-kube-api-access-7xk57\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925598 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925541    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-envoy\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925598 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925584    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c9e36e5-5009-4773-9b24-42c4d14fa329-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9e36e5-5009-4773-9b24-42c4d14fa329-istiod-ca-cert\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925730 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925631    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-credential-socket\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925730 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925704    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-token\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-token\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925745    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-socket" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:42.925838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925815    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-certs\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925946 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925846    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-data\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.925946 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925877    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-podinfo\") pod \"2c9e36e5-5009-4773-9b24-42c4d14fa329\" (UID: \"2c9e36e5-5009-4773-9b24-42c4d14fa329\") "
Nov 03 23:01:42.926056 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.925971    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-credential-socket" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:42.926185 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.926101    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-data" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:42.926185 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.926136    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c9e36e5-5009-4773-9b24-42c4d14fa329-istiod-ca-cert" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:01:42.926783 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.926758    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:42.926895 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.926790    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9e36e5-5009-4773-9b24-42c4d14fa329-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:42.926895 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.926807    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:42.926895 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.926823    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:42.927181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.927156    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-certs" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:42.928111 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.928093    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-kube-api-access-7xk57" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "kube-api-access-7xk57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:42.928265 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.928238    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-token" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:42.928265 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.928240    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-envoy" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:42.928447 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:42.928271    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-podinfo" pod "2c9e36e5-5009-4773-9b24-42c4d14fa329" (UID: "2c9e36e5-5009-4773-9b24-42c4d14fa329"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:01:43.028613 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.028583    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:43.028613 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.028607    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:43.028613 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.028618    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:43.028849 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.028630    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xk57\" (UniqueName: \"kubernetes.io/projected/2c9e36e5-5009-4773-9b24-42c4d14fa329-kube-api-access-7xk57\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:43.028849 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.028638    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9e36e5-5009-4773-9b24-42c4d14fa329-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:01:43.185605 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.185575    5162 generic.go:356] "Generic (PLEG): container finished" podID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerID="d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef" exitCode=0
Nov 03 23:01:43.185811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.185620    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"
Nov 03 23:01:43.185811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.185660    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" event={"ID":"2c9e36e5-5009-4773-9b24-42c4d14fa329","Type":"ContainerDied","Data":"d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef"}
Nov 03 23:01:43.185811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.185730    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55" event={"ID":"2c9e36e5-5009-4773-9b24-42c4d14fa329","Type":"ContainerDied","Data":"978e27aa82e6093a800627c37030e0ca04c8ecdc1d18bf81b6aa5a9042656416"}
Nov 03 23:01:43.185811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.185762    5162 scope.go:122] "RemoveContainer" containerID="d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef"
Nov 03 23:01:43.197127 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.197110    5162 scope.go:122] "RemoveContainer" containerID="ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8"
Nov 03 23:01:43.206296 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.206276    5162 scope.go:122] "RemoveContainer" containerID="d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef"
Nov 03 23:01:43.206618 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:01:43.206598    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef\": container with ID starting with d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef not found: ID does not exist" containerID="d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef"
Nov 03 23:01:43.206717 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.206625    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef"} err="failed to get container status \"d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef\": rpc error: code = NotFound desc = could not find container \"d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef\": container with ID starting with d31ada90fc50ad01701e8b4c32001257a7a43eada96fb52448f6a1bf1c1f95ef not found: ID does not exist"
Nov 03 23:01:43.206717 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.206644    5162 scope.go:122] "RemoveContainer" containerID="ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8"
Nov 03 23:01:43.206957 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:01:43.206935    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8\": container with ID starting with ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8 not found: ID does not exist" containerID="ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8"
Nov 03 23:01:43.207037 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.206962    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8"} err="failed to get container status \"ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8\": rpc error: code = NotFound desc = could not find container \"ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8\": container with ID starting with ea9e58eaf2b20cfd0a92548f06fce8867246edd0aa58a1a353c0f7857181e9f8 not found: ID does not exist"
Nov 03 23:01:43.356913 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.356881    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"]
Nov 03 23:01:43.363014 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:43.362981    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55"]
Nov 03 23:01:44.855090 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:44.855059    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" path="/var/lib/kubelet/pods/2c9e36e5-5009-4773-9b24-42c4d14fa329/volumes"
Nov 03 23:01:46.356711 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:46.356668    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:46.461356 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:46.461322    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"]
Nov 03 23:01:46.512686 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:46.512654    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"]
Nov 03 23:01:46.515562 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:46.515536    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"]
Nov 03 23:01:46.678830 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:46.678794    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:48.627868 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:48.627833    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:50.930050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:50.930010    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:51.678758 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:51.678720    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:52.011233 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:52.011114    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:52.222831 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:52.222797    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:53.550314 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:53.550280    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"]
Nov 03 23:01:53.746681 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:53.746643    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:54.338683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:54.338647    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:54.540678 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:54.540639    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:01:56.763792 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:01:56.763758    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:01.482598 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.482554    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main" containerID="cri-o://9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba" gracePeriod=45
Nov 03 23:02:01.532973 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.532925    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main" containerID="cri-o://efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77" gracePeriod=45
Nov 03 23:02:01.535284 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.535254    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main" containerID="cri-o://633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2" gracePeriod=45
Nov 03 23:02:01.678321 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.678217    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:01.870520 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.870489    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 23:02:01.886437 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.885636    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:01.907340 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.907313    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 23:02:01.938458 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.938431    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 23:02:01.962449 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962374    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-home\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-home\") pod \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") "
Nov 03 23:02:01.962636 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962532    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tls-certs\" (UniqueName: \"kubernetes.io/secret/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tls-certs\") pod \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") "
Nov 03 23:02:01.962636 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962595    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-dshm\") pod \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") "
Nov 03 23:02:01.962766 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962650    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-model-cache\") pod \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") "
Nov 03 23:02:01.962766 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962682    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-home" pod "c87f5c2c-0676-46ec-8512-8140bcaf92a8" (UID: "c87f5c2c-0676-46ec-8512-8140bcaf92a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.962766 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962691    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-home\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-home\") pod \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") "
Nov 03 23:02:01.962923 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962822    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tmp-dir\") pod \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") "
Nov 03 23:02:01.962923 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962901    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-dshm\") pod \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") "
Nov 03 23:02:01.963026 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.962935    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c87f5c2c-0676-46ec-8512-8140bcaf92a8-kube-api-access-v226s\" (UniqueName: \"kubernetes.io/projected/c87f5c2c-0676-46ec-8512-8140bcaf92a8-kube-api-access-v226s\") pod \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") "
Nov 03 23:02:01.963026 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.963011    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tmp-dir\") pod \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") "
Nov 03 23:02:01.963135 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.963119    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tls-certs\" (UniqueName: \"kubernetes.io/secret/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tls-certs\") pod \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\" (UID: \"c87f5c2c-0676-46ec-8512-8140bcaf92a8\") "
Nov 03 23:02:01.963210 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.963192    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-kube-api-access-wjdcp\" (UniqueName: \"kubernetes.io/projected/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-kube-api-access-wjdcp\") pod \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") "
Nov 03 23:02:01.963294 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.963278    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-model-cache\") pod \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\" (UID: \"d520739d-3f3e-4ce3-add4-4dcf6a03a30a\") "
Nov 03 23:02:01.963492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.963466    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-home" pod "d520739d-3f3e-4ce3-add4-4dcf6a03a30a" (UID: "d520739d-3f3e-4ce3-add4-4dcf6a03a30a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.963716 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.963694    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-model-cache" pod "d520739d-3f3e-4ce3-add4-4dcf6a03a30a" (UID: "d520739d-3f3e-4ce3-add4-4dcf6a03a30a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.963949 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.963927    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tmp-dir" pod "c87f5c2c-0676-46ec-8512-8140bcaf92a8" (UID: "c87f5c2c-0676-46ec-8512-8140bcaf92a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.964204 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.964182    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:01.964285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.964220    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:01.964285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.964235    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:01.964285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.964247    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:01.964492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.964440    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-model-cache" pod "c87f5c2c-0676-46ec-8512-8140bcaf92a8" (UID: "c87f5c2c-0676-46ec-8512-8140bcaf92a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.965054 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.965032    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tmp-dir" pod "d520739d-3f3e-4ce3-add4-4dcf6a03a30a" (UID: "d520739d-3f3e-4ce3-add4-4dcf6a03a30a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.966751 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.966719    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tls-certs" pod "d520739d-3f3e-4ce3-add4-4dcf6a03a30a" (UID: "d520739d-3f3e-4ce3-add4-4dcf6a03a30a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:01.967918 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.967862    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-kube-api-access-wjdcp" pod "d520739d-3f3e-4ce3-add4-4dcf6a03a30a" (UID: "d520739d-3f3e-4ce3-add4-4dcf6a03a30a"). InnerVolumeSpecName "kube-api-access-wjdcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:01.968015 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.967946    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-dshm" pod "c87f5c2c-0676-46ec-8512-8140bcaf92a8" (UID: "c87f5c2c-0676-46ec-8512-8140bcaf92a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.968096 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.968030    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-dshm" pod "d520739d-3f3e-4ce3-add4-4dcf6a03a30a" (UID: "d520739d-3f3e-4ce3-add4-4dcf6a03a30a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:01.969676 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.969650    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tls-certs" pod "c87f5c2c-0676-46ec-8512-8140bcaf92a8" (UID: "c87f5c2c-0676-46ec-8512-8140bcaf92a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:01.969832 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:01.969811    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c87f5c2c-0676-46ec-8512-8140bcaf92a8-kube-api-access-v226s" pod "c87f5c2c-0676-46ec-8512-8140bcaf92a8" (UID: "c87f5c2c-0676-46ec-8512-8140bcaf92a8"). InnerVolumeSpecName "kube-api-access-v226s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:02.065673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.065594    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/876c7542-5da1-42d2-8c62-57d41103f043-kube-api-access-ph64m\" (UniqueName: \"kubernetes.io/projected/876c7542-5da1-42d2-8c62-57d41103f043-kube-api-access-ph64m\") pod \"876c7542-5da1-42d2-8c62-57d41103f043\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") "
Nov 03 23:02:02.065855 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.065733    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-home\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-home\") pod \"876c7542-5da1-42d2-8c62-57d41103f043\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") "
Nov 03 23:02:02.065855 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.065753    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-tmp-dir\") pod \"876c7542-5da1-42d2-8c62-57d41103f043\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") "
Nov 03 23:02:02.065855 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.065803    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-model-cache\") pod \"876c7542-5da1-42d2-8c62-57d41103f043\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") "
Nov 03 23:02:02.065855 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.065852    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-dshm\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-dshm\") pod \"876c7542-5da1-42d2-8c62-57d41103f043\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") "
Nov 03 23:02:02.066091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.065926    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/876c7542-5da1-42d2-8c62-57d41103f043-tls-certs\" (UniqueName: \"kubernetes.io/secret/876c7542-5da1-42d2-8c62-57d41103f043-tls-certs\") pod \"876c7542-5da1-42d2-8c62-57d41103f043\" (UID: \"876c7542-5da1-42d2-8c62-57d41103f043\") "
Nov 03 23:02:02.066091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066062    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-home" pod "876c7542-5da1-42d2-8c62-57d41103f043" (UID: "876c7542-5da1-42d2-8c62-57d41103f043"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:02.066091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066039    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-tmp-dir" pod "876c7542-5da1-42d2-8c62-57d41103f043" (UID: "876c7542-5da1-42d2-8c62-57d41103f043"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:02.066232 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066120    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-model-cache" pod "876c7542-5da1-42d2-8c62-57d41103f043" (UID: "876c7542-5da1-42d2-8c62-57d41103f043"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:02.066437 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066420    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066437    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066448    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87f5c2c-0676-46ec-8512-8140bcaf92a8-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066457    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066466    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v226s\" (UniqueName: \"kubernetes.io/projected/c87f5c2c-0676-46ec-8512-8140bcaf92a8-kube-api-access-v226s\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066478    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066490    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c87f5c2c-0676-46ec-8512-8140bcaf92a8-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066501    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjdcp\" (UniqueName: \"kubernetes.io/projected/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-kube-api-access-wjdcp\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066509    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066521    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.066548 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.066535    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d520739d-3f3e-4ce3-add4-4dcf6a03a30a-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.068067 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.068038    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876c7542-5da1-42d2-8c62-57d41103f043-kube-api-access-ph64m" pod "876c7542-5da1-42d2-8c62-57d41103f043" (UID: "876c7542-5da1-42d2-8c62-57d41103f043"). InnerVolumeSpecName "kube-api-access-ph64m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:02.068166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.068132    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/876c7542-5da1-42d2-8c62-57d41103f043-tls-certs" pod "876c7542-5da1-42d2-8c62-57d41103f043" (UID: "876c7542-5da1-42d2-8c62-57d41103f043"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:02.068166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.068149    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-dshm" pod "876c7542-5da1-42d2-8c62-57d41103f043" (UID: "876c7542-5da1-42d2-8c62-57d41103f043"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:02.088879 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.088851    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:02.167808 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.167775    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph64m\" (UniqueName: \"kubernetes.io/projected/876c7542-5da1-42d2-8c62-57d41103f043-kube-api-access-ph64m\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.167808 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.167805    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/876c7542-5da1-42d2-8c62-57d41103f043-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.168010 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.167819    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/876c7542-5da1-42d2-8c62-57d41103f043-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:02.343262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.343167    5162 generic.go:356] "Generic (PLEG): container finished" podID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerID="efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77" exitCode=0
Nov 03 23:02:02.343262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.343217    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"
Nov 03 23:02:02.343262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.343245    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" event={"ID":"d520739d-3f3e-4ce3-add4-4dcf6a03a30a","Type":"ContainerDied","Data":"efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77"}
Nov 03 23:02:02.343571 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.343282    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692" event={"ID":"d520739d-3f3e-4ce3-add4-4dcf6a03a30a","Type":"ContainerDied","Data":"d4aeb70398bde5d43732805aa3288fee55d45269f97d617f6f30c87d20c82198"}
Nov 03 23:02:02.343571 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.343304    5162 scope.go:122] "RemoveContainer" containerID="efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77"
Nov 03 23:02:02.344709 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.344685    5162 generic.go:356] "Generic (PLEG): container finished" podID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerID="9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba" exitCode=0
Nov 03 23:02:02.344822 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.344716    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"
Nov 03 23:02:02.344822 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.344784    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" event={"ID":"c87f5c2c-0676-46ec-8512-8140bcaf92a8","Type":"ContainerDied","Data":"9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba"}
Nov 03 23:02:02.344822 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.344819    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t" event={"ID":"c87f5c2c-0676-46ec-8512-8140bcaf92a8","Type":"ContainerDied","Data":"30777f6ad881356af3a3ec3a556624bb4319a661e394f6d6674b74ad49454d18"}
Nov 03 23:02:02.352256 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.352117    5162 generic.go:356] "Generic (PLEG): container finished" podID="876c7542-5da1-42d2-8c62-57d41103f043" containerID="633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2" exitCode=0
Nov 03 23:02:02.352256 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.352187    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"
Nov 03 23:02:02.352256 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.352199    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" event={"ID":"876c7542-5da1-42d2-8c62-57d41103f043","Type":"ContainerDied","Data":"633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2"}
Nov 03 23:02:02.352256 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.352230    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2" event={"ID":"876c7542-5da1-42d2-8c62-57d41103f043","Type":"ContainerDied","Data":"cc8939a894749ca9c6bde8859039f41b1028a118f7f77284ef10f5947b14f002"}
Nov 03 23:02:02.359900 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.359876    5162 scope.go:122] "RemoveContainer" containerID="efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77"
Nov 03 23:02:02.360172 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:02.360154    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77\": container with ID starting with efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77 not found: ID does not exist" containerID="efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77"
Nov 03 23:02:02.360233 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.360180    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77"} err="failed to get container status \"efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77\": rpc error: code = NotFound desc = could not find container \"efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77\": container with ID starting with efee32b45dacca8644bc749e379cae87347d8d395d84134070f016ffe5ffac77 not found: ID does not exist"
Nov 03 23:02:02.360233 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.360197    5162 scope.go:122] "RemoveContainer" containerID="9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba"
Nov 03 23:02:02.369655 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.369636    5162 scope.go:122] "RemoveContainer" containerID="9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba"
Nov 03 23:02:02.369945 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:02.369927    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba\": container with ID starting with 9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba not found: ID does not exist" containerID="9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba"
Nov 03 23:02:02.369993 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.369954    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba"} err="failed to get container status \"9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba\": rpc error: code = NotFound desc = could not find container \"9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba\": container with ID starting with 9a2e95cad74f91fda74a945cf7d38f378552a4c61dc81eac33382d28299fa5ba not found: ID does not exist"
Nov 03 23:02:02.369993 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.369973    5162 scope.go:122] "RemoveContainer" containerID="633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2"
Nov 03 23:02:02.378705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.378686    5162 scope.go:122] "RemoveContainer" containerID="633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2"
Nov 03 23:02:02.378965 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:02.378946    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2\": container with ID starting with 633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2 not found: ID does not exist" containerID="633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2"
Nov 03 23:02:02.379021 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.378972    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2"} err="failed to get container status \"633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2\": rpc error: code = NotFound desc = could not find container \"633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2\": container with ID starting with 633fd9445d89936ec42ce35b21e2e386c6342d701b48a11b8b8c79f443ca55c2 not found: ID does not exist"
Nov 03 23:02:02.512660 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.512622    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"]
Nov 03 23:02:02.516794 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.516769    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/premium-w1-545b5d-kserve-5dc8ff84bd-4r692"]
Nov 03 23:02:02.628965 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.628842    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"]
Nov 03 23:02:02.632880 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.632853    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/distinct-2-w1-545b5d-kserve-667b564f6-gsj8t"]
Nov 03 23:02:02.782415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.773446    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"]
Nov 03 23:02:02.802187 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.802141    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-545b5d/simulator-w1-545b5d-kserve-b689679d9-lqmv2"]
Nov 03 23:02:02.856249 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.856219    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="876c7542-5da1-42d2-8c62-57d41103f043" path="/var/lib/kubelet/pods/876c7542-5da1-42d2-8c62-57d41103f043/volumes"
Nov 03 23:02:02.856742 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.856728    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" path="/var/lib/kubelet/pods/c87f5c2c-0676-46ec-8512-8140bcaf92a8/volumes"
Nov 03 23:02:02.857115 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:02.857103    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" path="/var/lib/kubelet/pods/d520739d-3f3e-4ce3-add4-4dcf6a03a30a/volumes"
Nov 03 23:02:06.249365 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:06.249325    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:07.156331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:07.156230    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:07.423258 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:07.423220    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:07.626639 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:07.626603    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:08.571581 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.571538    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main" containerID="cri-o://fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5" gracePeriod=45
Nov 03 23:02:08.836238 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.836182    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 23:02:08.968483 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.968444    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96987f24-3f13-4d94-8754-208bbeb8844e-tls-certs\" (UniqueName: \"kubernetes.io/secret/96987f24-3f13-4d94-8754-208bbeb8844e-tls-certs\") pod \"96987f24-3f13-4d94-8754-208bbeb8844e\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") "
Nov 03 23:02:08.968691 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.968511    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-model-cache\") pod \"96987f24-3f13-4d94-8754-208bbeb8844e\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") "
Nov 03 23:02:08.968691 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.968547    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-tmp-dir\") pod \"96987f24-3f13-4d94-8754-208bbeb8844e\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") "
Nov 03 23:02:08.968691 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.968581    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-dshm\") pod \"96987f24-3f13-4d94-8754-208bbeb8844e\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") "
Nov 03 23:02:08.968691 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.968675    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96987f24-3f13-4d94-8754-208bbeb8844e-kube-api-access-zh6gp\" (UniqueName: \"kubernetes.io/projected/96987f24-3f13-4d94-8754-208bbeb8844e-kube-api-access-zh6gp\") pod \"96987f24-3f13-4d94-8754-208bbeb8844e\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") "
Nov 03 23:02:08.968914 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.968729    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-home\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-home\") pod \"96987f24-3f13-4d94-8754-208bbeb8844e\" (UID: \"96987f24-3f13-4d94-8754-208bbeb8844e\") "
Nov 03 23:02:08.968914 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.968837    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-model-cache" pod "96987f24-3f13-4d94-8754-208bbeb8844e" (UID: "96987f24-3f13-4d94-8754-208bbeb8844e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:08.969057 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.969035    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-tmp-dir" pod "96987f24-3f13-4d94-8754-208bbeb8844e" (UID: "96987f24-3f13-4d94-8754-208bbeb8844e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:08.969245 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.969226    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-home" pod "96987f24-3f13-4d94-8754-208bbeb8844e" (UID: "96987f24-3f13-4d94-8754-208bbeb8844e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:08.969610 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.969587    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:08.969900 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.969877    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:08.970058 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.970019    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:08.972107 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.972077    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-dshm" pod "96987f24-3f13-4d94-8754-208bbeb8844e" (UID: "96987f24-3f13-4d94-8754-208bbeb8844e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:08.972355 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.972332    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96987f24-3f13-4d94-8754-208bbeb8844e-tls-certs" pod "96987f24-3f13-4d94-8754-208bbeb8844e" (UID: "96987f24-3f13-4d94-8754-208bbeb8844e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:08.972467 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:08.972384    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96987f24-3f13-4d94-8754-208bbeb8844e-kube-api-access-zh6gp" pod "96987f24-3f13-4d94-8754-208bbeb8844e" (UID: "96987f24-3f13-4d94-8754-208bbeb8844e"). InnerVolumeSpecName "kube-api-access-zh6gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:09.070828 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.070793    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96987f24-3f13-4d94-8754-208bbeb8844e-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:09.070828 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.070827    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh6gp\" (UniqueName: \"kubernetes.io/projected/96987f24-3f13-4d94-8754-208bbeb8844e-kube-api-access-zh6gp\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:09.071015 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.070843    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96987f24-3f13-4d94-8754-208bbeb8844e-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:09.399106 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.399070    5162 generic.go:356] "Generic (PLEG): container finished" podID="96987f24-3f13-4d94-8754-208bbeb8844e" containerID="fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5" exitCode=0
Nov 03 23:02:09.399106 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.399110    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" event={"ID":"96987f24-3f13-4d94-8754-208bbeb8844e","Type":"ContainerDied","Data":"fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5"}
Nov 03 23:02:09.399331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.399112    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"
Nov 03 23:02:09.399331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.399150    5162 scope.go:122] "RemoveContainer" containerID="fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5"
Nov 03 23:02:09.399331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.399138    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5" event={"ID":"96987f24-3f13-4d94-8754-208bbeb8844e","Type":"ContainerDied","Data":"0f2302d9204cfe41cb970f91aa66361eecc546f645dd90da8ddb30255658fd13"}
Nov 03 23:02:09.409017 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.408998    5162 scope.go:122] "RemoveContainer" containerID="fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5"
Nov 03 23:02:09.409282 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:09.409253    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5\": container with ID starting with fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5 not found: ID does not exist" containerID="fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5"
Nov 03 23:02:09.409348 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.409291    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5"} err="failed to get container status \"fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5\": rpc error: code = NotFound desc = could not find container \"fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5\": container with ID starting with fd72fe2ee93c1eb768ff35fcd8d638986c54a80b1843be6ca3dad5e221e4c1f5 not found: ID does not exist"
Nov 03 23:02:09.571785 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.571755    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"]
Nov 03 23:02:09.623114 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:09.623085    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-ee74cb/simulator-w2-ee74cb-kserve-5554b74f97-zktr5"]
Nov 03 23:02:10.087207 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:10.087168    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:02:10.854799 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:10.854768    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" path="/var/lib/kubelet/pods/96987f24-3f13-4d94-8754-208bbeb8844e/volumes"
Nov 03 23:02:12.064993 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:12.064956    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:12.313325 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:12.313288    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:12.767427 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:12.767368    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:17.130677 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:17.128721    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:17.545528 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:17.545496    5162 scope.go:122] "RemoveContainer" containerID="bcac9ce0c2d3001683283ec14a9181bc5d107850782c4f568bb0248d1901b102"
Nov 03 23:02:20.625972 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:20.625930    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:21.131025 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.130991    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"]
Nov 03 23:02:21.131271 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.131231    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk" podUID="d895f971-4533-4101-b67b-1fec8a211ece" containerName="maas-api" containerID="cri-o://bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58" gracePeriod=30
Nov 03 23:02:21.443646 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.443619    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 23:02:21.532869 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.532835    5162 generic.go:356] "Generic (PLEG): container finished" podID="d895f971-4533-4101-b67b-1fec8a211ece" containerID="bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58" exitCode=0
Nov 03 23:02:21.533225 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.533051    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"
Nov 03 23:02:21.533225 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.533112    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk" event={"ID":"d895f971-4533-4101-b67b-1fec8a211ece","Type":"ContainerDied","Data":"bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58"}
Nov 03 23:02:21.533225 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.533164    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk" event={"ID":"d895f971-4533-4101-b67b-1fec8a211ece","Type":"ContainerDied","Data":"3c79a3d166017e3b6c9ec7a91cab55da42bc4b612445cd45c27b72bca63f1a19"}
Nov 03 23:02:21.533225 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.533196    5162 scope.go:122] "RemoveContainer" containerID="bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58"
Nov 03 23:02:21.566983 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.566959    5162 scope.go:122] "RemoveContainer" containerID="bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58"
Nov 03 23:02:21.567443 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:21.567386    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58\": container with ID starting with bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58 not found: ID does not exist" containerID="bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58"
Nov 03 23:02:21.567570 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.567453    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58"} err="failed to get container status \"bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58\": rpc error: code = NotFound desc = could not find container \"bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58\": container with ID starting with bc583dc7e3105fcfa323e062bc6484cda7a0439c43ee307039472ac5a293be58 not found: ID does not exist"
Nov 03 23:02:21.652264 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.652235    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d895f971-4533-4101-b67b-1fec8a211ece-kube-api-access-hq7vh\" (UniqueName: \"kubernetes.io/projected/d895f971-4533-4101-b67b-1fec8a211ece-kube-api-access-hq7vh\") pod \"d895f971-4533-4101-b67b-1fec8a211ece\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") "
Nov 03 23:02:21.652683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.652379    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-metrics-certs\") pod \"d895f971-4533-4101-b67b-1fec8a211ece\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") "
Nov 03 23:02:21.652683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.652489    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d895f971-4533-4101-b67b-1fec8a211ece-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d895f971-4533-4101-b67b-1fec8a211ece-openshift-service-ca\") pod \"d895f971-4533-4101-b67b-1fec8a211ece\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") "
Nov 03 23:02:21.652683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.652541    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls\") pod \"d895f971-4533-4101-b67b-1fec8a211ece\" (UID: \"d895f971-4533-4101-b67b-1fec8a211ece\") "
Nov 03 23:02:21.652904 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.652874    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d895f971-4533-4101-b67b-1fec8a211ece-openshift-service-ca" pod "d895f971-4533-4101-b67b-1fec8a211ece" (UID: "d895f971-4533-4101-b67b-1fec8a211ece"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:02:21.653064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.653047    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d895f971-4533-4101-b67b-1fec8a211ece-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:21.654505 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.654486    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls" pod "d895f971-4533-4101-b67b-1fec8a211ece" (UID: "d895f971-4533-4101-b67b-1fec8a211ece"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:21.654799 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.654779    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-metrics-certs" pod "d895f971-4533-4101-b67b-1fec8a211ece" (UID: "d895f971-4533-4101-b67b-1fec8a211ece"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:21.654863 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.654814    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d895f971-4533-4101-b67b-1fec8a211ece-kube-api-access-hq7vh" pod "d895f971-4533-4101-b67b-1fec8a211ece" (UID: "d895f971-4533-4101-b67b-1fec8a211ece"). InnerVolumeSpecName "kube-api-access-hq7vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:21.753867 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.753830    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:21.753867 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.753865    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d895f971-4533-4101-b67b-1fec8a211ece-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:21.753867 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.753876    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq7vh\" (UniqueName: \"kubernetes.io/projected/d895f971-4533-4101-b67b-1fec8a211ece-kube-api-access-hq7vh\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:21.958409 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.958367    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"]
Nov 03 23:02:21.965205 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:21.965180    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-961054-86d699db45-47wpk"]
Nov 03 23:02:22.410811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:22.410760    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:22.855029 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:22.854995    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d895f971-4533-4101-b67b-1fec8a211ece" path="/var/lib/kubelet/pods/d895f971-4533-4101-b67b-1fec8a211ece/volumes"
Nov 03 23:02:22.889337 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:22.889310    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:23.194959 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:23.193292    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:23.404530 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:23.404495    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:23.828875 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:23.828835    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:24.483414 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:24.483364    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:24.683806 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:24.683767    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:24.958018 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:24.957970    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:28.255358 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:28.255279    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:29.712285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:29.712238    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:29.975921 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:29.975805    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:30.008132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:30.008098    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"]
Nov 03 23:02:30.008417 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:30.008367    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy" containerID="cri-o://ad9fa409ab7df5b20e323c0794855bb326e0cd66dc9ecf131bbab06b2c2ef45e" gracePeriod=30
Nov 03 23:02:30.213938 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:30.213900    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:31.892762 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:31.892723    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:32.140361 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:32.140329    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:33.231417 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:33.220164    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:35.626598 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.626563    5162 generic.go:356] "Generic (PLEG): container finished" podID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerID="ad9fa409ab7df5b20e323c0794855bb326e0cd66dc9ecf131bbab06b2c2ef45e" exitCode=0
Nov 03 23:02:35.627046 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.626734    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" event={"ID":"91745d21-1539-4c6a-a38d-250b69eb48ef","Type":"ContainerDied","Data":"ad9fa409ab7df5b20e323c0794855bb326e0cd66dc9ecf131bbab06b2c2ef45e"}
Nov 03 23:02:35.627046 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.626779    5162 scope.go:122] "RemoveContainer" containerID="a905ec820d03d51e7b246db1f45437c187409a94e2c957a4b42193040da760c6"
Nov 03 23:02:35.680360 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.680338    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 23:02:35.744814 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.744782    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-socket\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.744974 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.744828    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91745d21-1539-4c6a-a38d-250b69eb48ef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91745d21-1539-4c6a-a38d-250b69eb48ef-istiod-ca-cert\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.744974 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.744876    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-certs\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.744974 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.744920    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-kube-api-access-f8d4h\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-kube-api-access-f8d4h\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.745143 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745009    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-podinfo\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.745143 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745063    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-socket" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:35.745143 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745078    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-envoy\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.745302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745184    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-certs" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:35.745302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745250    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-data\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.745302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745273    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91745d21-1539-4c6a-a38d-250b69eb48ef-istiod-ca-cert" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:02:35.745302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745293    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-credential-socket\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.745581 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745362    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-token\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-token\") pod \"91745d21-1539-4c6a-a38d-250b69eb48ef\" (UID: \"91745d21-1539-4c6a-a38d-250b69eb48ef\") "
Nov 03 23:02:35.745581 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745473    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-data" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:35.745692 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745641    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-credential-socket" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:35.745971 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745948    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.746054 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745976    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.746054 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.745991    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.746054 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.746006    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91745d21-1539-4c6a-a38d-250b69eb48ef-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.746054 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.746019    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.747455 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.747342    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-envoy" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:02:35.747455 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.747439    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-token" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:35.747625 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.747453    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-kube-api-access-f8d4h" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "kube-api-access-f8d4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:35.747625 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.747470    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-podinfo" pod "91745d21-1539-4c6a-a38d-250b69eb48ef" (UID: "91745d21-1539-4c6a-a38d-250b69eb48ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:02:35.847275 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.847197    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.847275 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.847225    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8d4h\" (UniqueName: \"kubernetes.io/projected/91745d21-1539-4c6a-a38d-250b69eb48ef-kube-api-access-f8d4h\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.847275 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.847234    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:35.847275 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:35.847244    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91745d21-1539-4c6a-a38d-250b69eb48ef-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:36.635381 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:36.635350    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"
Nov 03 23:02:36.635865 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:36.635411    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7" event={"ID":"91745d21-1539-4c6a-a38d-250b69eb48ef","Type":"ContainerDied","Data":"2fd05cc603dc1d29b0b99472b05b33d895062d7813bb921f43b7d2d85f82e469"}
Nov 03 23:02:36.635865 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:36.635482    5162 scope.go:122] "RemoveContainer" containerID="ad9fa409ab7df5b20e323c0794855bb326e0cd66dc9ecf131bbab06b2c2ef45e"
Nov 03 23:02:36.806361 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:36.806332    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"]
Nov 03 23:02:36.858034 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:36.858001    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7"]
Nov 03 23:02:38.164502 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:38.163368    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:38.854741 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:38.854711    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" path="/var/lib/kubelet/pods/91745d21-1539-4c6a-a38d-250b69eb48ef/volumes"
Nov 03 23:02:38.914856 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:38.914819    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:40.083472 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:40.083437    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:02:42.276585 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.276551    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"]
Nov 03 23:02:42.277338 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.277283    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww" podUID="063a642b-b204-45bb-893b-0e685d5dabc8" containerName="maas-api" containerID="cri-o://beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed" gracePeriod=30
Nov 03 23:02:42.516114 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.516072    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:42.558467 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.558437    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 23:02:42.703424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.702682    5162 generic.go:356] "Generic (PLEG): container finished" podID="063a642b-b204-45bb-893b-0e685d5dabc8" containerID="beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed" exitCode=0
Nov 03 23:02:42.703424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.702739    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww" event={"ID":"063a642b-b204-45bb-893b-0e685d5dabc8","Type":"ContainerDied","Data":"beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed"}
Nov 03 23:02:42.703424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.702788    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww" event={"ID":"063a642b-b204-45bb-893b-0e685d5dabc8","Type":"ContainerDied","Data":"da5234af0c2fc87c328a6dff121ede3a20185be9b15f935d964d83748a4a6ec9"}
Nov 03 23:02:42.703424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.702816    5162 scope.go:122] "RemoveContainer" containerID="beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed"
Nov 03 23:02:42.703424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.702964    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"
Nov 03 23:02:42.716656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.716634    5162 scope.go:122] "RemoveContainer" containerID="beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed"
Nov 03 23:02:42.716983 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:02:42.716959    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed\": container with ID starting with beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed not found: ID does not exist" containerID="beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed"
Nov 03 23:02:42.717065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.716997    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed"} err="failed to get container status \"beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed\": rpc error: code = NotFound desc = could not find container \"beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed\": container with ID starting with beff7a671db864facb99a7ddae739d49a4eb9681048f51e168b6cd8c813d3aed not found: ID does not exist"
Nov 03 23:02:42.734399 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.734368    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/063a642b-b204-45bb-893b-0e685d5dabc8-kube-api-access-k4c9j\" (UniqueName: \"kubernetes.io/projected/063a642b-b204-45bb-893b-0e685d5dabc8-kube-api-access-k4c9j\") pod \"063a642b-b204-45bb-893b-0e685d5dabc8\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") "
Nov 03 23:02:42.734593 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.734573    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls\") pod \"063a642b-b204-45bb-893b-0e685d5dabc8\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") "
Nov 03 23:02:42.734700 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.734636    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-metrics-certs\") pod \"063a642b-b204-45bb-893b-0e685d5dabc8\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") "
Nov 03 23:02:42.734819 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.734801    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/063a642b-b204-45bb-893b-0e685d5dabc8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/063a642b-b204-45bb-893b-0e685d5dabc8-openshift-service-ca\") pod \"063a642b-b204-45bb-893b-0e685d5dabc8\" (UID: \"063a642b-b204-45bb-893b-0e685d5dabc8\") "
Nov 03 23:02:42.735161 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.735136    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063a642b-b204-45bb-893b-0e685d5dabc8-openshift-service-ca" pod "063a642b-b204-45bb-893b-0e685d5dabc8" (UID: "063a642b-b204-45bb-893b-0e685d5dabc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:02:42.735313 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.735296    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/063a642b-b204-45bb-893b-0e685d5dabc8-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:42.736620 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.736596    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063a642b-b204-45bb-893b-0e685d5dabc8-kube-api-access-k4c9j" pod "063a642b-b204-45bb-893b-0e685d5dabc8" (UID: "063a642b-b204-45bb-893b-0e685d5dabc8"). InnerVolumeSpecName "kube-api-access-k4c9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:42.736830 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.736817    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls" pod "063a642b-b204-45bb-893b-0e685d5dabc8" (UID: "063a642b-b204-45bb-893b-0e685d5dabc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:42.737144 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.737121    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-metrics-certs" pod "063a642b-b204-45bb-893b-0e685d5dabc8" (UID: "063a642b-b204-45bb-893b-0e685d5dabc8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:02:42.837024 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.836942    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:42.837024 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.836971    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4c9j\" (UniqueName: \"kubernetes.io/projected/063a642b-b204-45bb-893b-0e685d5dabc8-kube-api-access-k4c9j\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:42.837024 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:42.836981    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/063a642b-b204-45bb-893b-0e685d5dabc8-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:02:43.120896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:43.120809    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"]
Nov 03 23:02:43.124930 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:43.124907    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-a03fee-59c7b68b67-dwcww"]
Nov 03 23:02:43.993721 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:43.993677    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:44.288737 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:44.286468    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:44.490664 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:44.490621    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:44.856210 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:44.856182    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="063a642b-b204-45bb-893b-0e685d5dabc8" path="/var/lib/kubelet/pods/063a642b-b204-45bb-893b-0e685d5dabc8/volumes"
Nov 03 23:02:44.904610 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:44.904578    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:48.323436 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:48.321088    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:49.588248 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:49.587571    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:51.399090 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:51.399057    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:51.639668 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:51.639631    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:51.847643 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:51.847610    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:52.257437 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:52.257382    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:54.585356 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:54.585311    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:55.027466 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:55.025954    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:56.101741 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:56.101696    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:56.425038 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:56.425004    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:02:57.152880 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:57.152840    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"]
Nov 03 23:02:57.314196 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:02:57.314153    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:00.400658 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:00.400616    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:00.603137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:00.603102    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:04.427296 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:04.427259    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:04.626370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:04.626328    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:05.037066 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:05.037031    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:06.221252 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:06.221210    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:06.639198 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:06.639113    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:06.858727 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:06.858691    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:07.062847 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.062812    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:07.261098 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.261062    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:07.662022 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.661988    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:07.839559 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.839527    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"]
Nov 03 23:03:07.840036 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840022    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.840092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840038    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.840092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840052    5162 cpu_manager.go:414] "containerMap: removing container" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy"
Nov 03 23:03:07.840092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840057    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy"
Nov 03 23:03:07.840092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840068    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy"
Nov 03 23:03:07.840092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840074    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy"
Nov 03 23:03:07.840092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840083    5162 cpu_manager.go:414] "containerMap: removing container" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main"
Nov 03 23:03:07.840092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840087    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840096    5162 cpu_manager.go:414] "containerMap: removing container" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840101    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840109    5162 cpu_manager.go:414] "containerMap: removing container" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840113    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840119    5162 cpu_manager.go:414] "containerMap: removing container" podUID="063a642b-b204-45bb-893b-0e685d5dabc8" containerName="maas-api"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840123    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="063a642b-b204-45bb-893b-0e685d5dabc8" containerName="maas-api"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840128    5162 cpu_manager.go:414] "containerMap: removing container" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840132    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840144    5162 cpu_manager.go:414] "containerMap: removing container" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840148    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840154    5162 cpu_manager.go:414] "containerMap: removing container" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840158    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840171    5162 cpu_manager.go:414] "containerMap: removing container" podUID="d895f971-4533-4101-b67b-1fec8a211ece" containerName="maas-api"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840175    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d895f971-4533-4101-b67b-1fec8a211ece" containerName="maas-api"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840183    5162 cpu_manager.go:414] "containerMap: removing container" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840188    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840193    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840197    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840203    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840207    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840213    5162 cpu_manager.go:414] "containerMap: removing container" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840218    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840224    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840228    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840236    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840240    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840252    5162 cpu_manager.go:414] "containerMap: removing container" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840256    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840263    5162 cpu_manager.go:414] "containerMap: removing container" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840267    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840273    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840277    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840283    5162 cpu_manager.go:414] "containerMap: removing container" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840288    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840298    5162 cpu_manager.go:414] "containerMap: removing container" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840302    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840310    5162 cpu_manager.go:414] "containerMap: removing container" podUID="07b6e3a4-895b-4344-8897-22455346de35" containerName="payload-pre-processing"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840315    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07b6e3a4-895b-4344-8897-22455346de35" containerName="payload-pre-processing"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840322    5162 cpu_manager.go:414] "containerMap: removing container" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840326    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840333    5162 cpu_manager.go:414] "containerMap: removing container" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840337    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main"
Nov 03 23:03:07.840323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840343    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840348    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840354    5162 cpu_manager.go:414] "containerMap: removing container" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840358    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840366    5162 cpu_manager.go:414] "containerMap: removing container" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840370    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840378    5162 cpu_manager.go:414] "containerMap: removing container" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840382    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840467    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840478    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840485    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="876c7542-5da1-42d2-8c62-57d41103f043" containerName="main"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840492    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840498    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="91745d21-1539-4c6a-a38d-250b69eb48ef" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840504    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="063a642b-b204-45bb-893b-0e685d5dabc8" containerName="maas-api"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840509    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eca86f5-f01a-4000-8118-27836d87ba36" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840515    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840520    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840526    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="d895f971-4533-4101-b67b-1fec8a211ece" containerName="maas-api"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840532    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="daea8b8c-88b2-41d7-a1cf-7b4fd4306e06" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840537    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840541    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840545    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="4783432e-dd9f-4416-b435-c6960145a39c" containerName="main"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840550    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d6d9588-1dfd-4247-ad1a-9cdd6f7d7051" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840555    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840560    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840565    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840571    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbdb117f-e199-4424-9674-69c4f71224d4" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840577    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840581    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="07b6e3a4-895b-4344-8897-22455346de35" containerName="payload-pre-processing"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840585    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="d520739d-3f3e-4ce3-add4-4dcf6a03a30a" containerName="main"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840590    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="c87f5c2c-0676-46ec-8512-8140bcaf92a8" containerName="main"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840595    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840600    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c9e36e5-5009-4773-9b24-42c4d14fa329" containerName="istio-proxy"
Nov 03 23:03:07.841673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.840605    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="96987f24-3f13-4d94-8754-208bbeb8844e" containerName="main"
Nov 03 23:03:07.844079 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.844064    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.893587 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.893533    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pg78\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-kube-api-access-7pg78\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.893759 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.893715    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-socket\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.894190 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.894161    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-token\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.894307 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.894266    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istiod-ca-cert\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.894373 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.894348    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-envoy\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.895498 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.895380    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-certs\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.895498 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.895449    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-podinfo\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.895498 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.895483    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-credential-socket\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.895694 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.895514    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-data\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:07.901664 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.901643    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"]
Nov 03 23:03:07.902545 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.902518    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:07.948811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:07.948792    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ac6633-openshift-default-dockercfg-74dnh\""
Nov 03 23:03:08.001719 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.001680    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-data\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.002045 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.002027    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pg78\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-kube-api-access-7pg78\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.002189 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.002173    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-socket\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.002370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.002354    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-token\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.003891 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.003866    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istiod-ca-cert\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.004112 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.004090    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-envoy\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.004352 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.004322    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-certs\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.004553 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.004532    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-podinfo\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.004710 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.004691    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-credential-socket\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.004948 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.004921    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-socket\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.005181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.005162    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-certs\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.005324 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.005293    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-credential-socket\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.005450 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.004466    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-data\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.009371 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.005756    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istiod-ca-cert\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.011738 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.011713    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-envoy\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.012377 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.012353    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-podinfo\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.066883 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.066854    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-token\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.067030 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.066945    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pg78\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-kube-api-access-7pg78\") pod \"e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") " pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.148233 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.148183    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:08.157982 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.157956    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:08.351966 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.351934    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:08.580572 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:03:08.580545    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013e4f6a_2cf6_4d66_8580_77c5e2359f53.slice/crio-98c8fdd49633422110a9dae4daf680e1ff71a76f7289221073532dc7012d0dc0 WatchSource:0}: Error finding container 98c8fdd49633422110a9dae4daf680e1ff71a76f7289221073532dc7012d0dc0: Status 404 returned error can't find the container with id 98c8fdd49633422110a9dae4daf680e1ff71a76f7289221073532dc7012d0dc0
Nov 03 23:03:08.583753 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.583711    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:03:08.583855 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.583803    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:03:08.583855 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.583845    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:03:08.584473 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.584375    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"]
Nov 03 23:03:08.865584 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.865486    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:08.866457 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.866431    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" event={"ID":"013e4f6a-2cf6-4d66-8580-77c5e2359f53","Type":"ContainerStarted","Data":"2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55"}
Nov 03 23:03:08.866580 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.866478    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" event={"ID":"013e4f6a-2cf6-4d66-8580-77c5e2359f53","Type":"ContainerStarted","Data":"98c8fdd49633422110a9dae4daf680e1ff71a76f7289221073532dc7012d0dc0"}
Nov 03 23:03:08.997452 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:08.997366    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podStartSLOduration=1.9973517570000001 podStartE2EDuration="1.997351757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:03:08.995693762 +0000 UTC m=+2392.260393783" watchObservedRunningTime="2026-11-03 23:03:08.997351757 +0000 UTC m=+2392.262051767"
Nov 03 23:03:09.158353 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:09.158273    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:09.163932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:09.163906    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:09.872896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:09.872862    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:09.873920 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:09.873901    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:03:10.095871 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:03:10.089422    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:03:11.879693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:11.879655    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:12.142228 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.142151    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:12.173613 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.173585    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main" containerID="cri-o://8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7" gracePeriod=45
Nov 03 23:03:12.443345 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.443321    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 23:03:12.570886 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.570856    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-dshm\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-dshm\") pod \"be81b953-ea9f-491b-be35-977a05fa2106\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") "
Nov 03 23:03:12.571074 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.570931    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be81b953-ea9f-491b-be35-977a05fa2106-kube-api-access-4x4cb\" (UniqueName: \"kubernetes.io/projected/be81b953-ea9f-491b-be35-977a05fa2106-kube-api-access-4x4cb\") pod \"be81b953-ea9f-491b-be35-977a05fa2106\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") "
Nov 03 23:03:12.571074 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.570971    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-home\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-home\") pod \"be81b953-ea9f-491b-be35-977a05fa2106\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") "
Nov 03 23:03:12.571203 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.571110    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be81b953-ea9f-491b-be35-977a05fa2106-tls-certs\" (UniqueName: \"kubernetes.io/secret/be81b953-ea9f-491b-be35-977a05fa2106-tls-certs\") pod \"be81b953-ea9f-491b-be35-977a05fa2106\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") "
Nov 03 23:03:12.571325 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.571286    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-home" pod "be81b953-ea9f-491b-be35-977a05fa2106" (UID: "be81b953-ea9f-491b-be35-977a05fa2106"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:12.571411 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.571355    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-model-cache\") pod \"be81b953-ea9f-491b-be35-977a05fa2106\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") "
Nov 03 23:03:12.571482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.571449    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-tmp-dir\") pod \"be81b953-ea9f-491b-be35-977a05fa2106\" (UID: \"be81b953-ea9f-491b-be35-977a05fa2106\") "
Nov 03 23:03:12.571906 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.571779    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-model-cache" pod "be81b953-ea9f-491b-be35-977a05fa2106" (UID: "be81b953-ea9f-491b-be35-977a05fa2106"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:12.571906 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.571875    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-tmp-dir" pod "be81b953-ea9f-491b-be35-977a05fa2106" (UID: "be81b953-ea9f-491b-be35-977a05fa2106"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:12.572082 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.572062    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:12.572132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.572089    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:12.572132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.572103    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:12.574786 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.574763    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be81b953-ea9f-491b-be35-977a05fa2106-tls-certs" pod "be81b953-ea9f-491b-be35-977a05fa2106" (UID: "be81b953-ea9f-491b-be35-977a05fa2106"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:03:12.574880 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.574827    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be81b953-ea9f-491b-be35-977a05fa2106-kube-api-access-4x4cb" pod "be81b953-ea9f-491b-be35-977a05fa2106" (UID: "be81b953-ea9f-491b-be35-977a05fa2106"). InnerVolumeSpecName "kube-api-access-4x4cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:12.575515 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.575493    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-dshm" pod "be81b953-ea9f-491b-be35-977a05fa2106" (UID: "be81b953-ea9f-491b-be35-977a05fa2106"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:12.673773 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.673749    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be81b953-ea9f-491b-be35-977a05fa2106-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:12.673773 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.673775    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x4cb\" (UniqueName: \"kubernetes.io/projected/be81b953-ea9f-491b-be35-977a05fa2106-kube-api-access-4x4cb\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:12.673969 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.673785    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be81b953-ea9f-491b-be35-977a05fa2106-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:12.891775 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.891747    5162 generic.go:356] "Generic (PLEG): container finished" podID="be81b953-ea9f-491b-be35-977a05fa2106" containerID="8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7" exitCode=0
Nov 03 23:03:12.892188 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.891791    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"
Nov 03 23:03:12.892188 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.891832    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" event={"ID":"be81b953-ea9f-491b-be35-977a05fa2106","Type":"ContainerDied","Data":"8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7"}
Nov 03 23:03:12.892188 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.891879    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn" event={"ID":"be81b953-ea9f-491b-be35-977a05fa2106","Type":"ContainerDied","Data":"f2297195291bccf18270a265678ae3389dbe5422c1b498abbc416b06e00edffb"}
Nov 03 23:03:12.892188 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.891897    5162 scope.go:122] "RemoveContainer" containerID="8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7"
Nov 03 23:03:12.901285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.901271    5162 scope.go:122] "RemoveContainer" containerID="8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7"
Nov 03 23:03:12.901558 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:03:12.901537    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7\": container with ID starting with 8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7 not found: ID does not exist" containerID="8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7"
Nov 03 23:03:12.901616 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:12.901569    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7"} err="failed to get container status \"8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7\": rpc error: code = NotFound desc = could not find container \"8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7\": container with ID starting with 8d11ea85bcbb9fae5fd62e1d3aef549cb6ba9a0bc90c93956cda4f2747a9c2d7 not found: ID does not exist"
Nov 03 23:03:13.064194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.064120    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"]
Nov 03 23:03:13.115804 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.115769    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-18c03a/distinct-w4-18c03a-kserve-845b7c549f-cpknn"]
Nov 03 23:03:13.231309 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231279    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"]
Nov 03 23:03:13.231789 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231774    5162 cpu_manager.go:414] "containerMap: removing container" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main"
Nov 03 23:03:13.231841 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231790    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main"
Nov 03 23:03:13.231841 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231808    5162 cpu_manager.go:414] "containerMap: removing container" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:13.231841 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231813    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:13.231932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231897    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="be81b953-ea9f-491b-be35-977a05fa2106" containerName="main"
Nov 03 23:03:13.231932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231906    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="a876d4eb-4282-4ae9-9726-63a18708508f" containerName="cleanup"
Nov 03 23:03:13.231932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231914    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f7b63bc-125d-4b96-b401-c4dfcc396922" containerName="cleanup"
Nov 03 23:03:13.231932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231923    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="6bb8a3d7-b5fa-4761-98f3-065f9365d34a" containerName="cleanup"
Nov 03 23:03:13.231932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.231929    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8cf17ac-f1dd-44dd-b3bb-627c61227327" containerName="cleanup"
Nov 03 23:03:13.236543 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.236522    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.294172 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.294134    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"]
Nov 03 23:03:13.384162 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384081    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2s9k\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-kube-api-access-h2s9k\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384162 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384148    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0cda92-bdf4-456b-8953-056663ce0359-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384387 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384276    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384387 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384326    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384519 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384459    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384519 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384489    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384545    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384664 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384648    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.384710 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.384689    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0cda92-bdf4-456b-8953-056663ce0359-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.485732 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485703    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2s9k\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-kube-api-access-h2s9k\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.485917 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485760    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0cda92-bdf4-456b-8953-056663ce0359-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.485917 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485791    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.485917 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485818    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.485917 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485864    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.485917 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485878    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.485917 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485896    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.486236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.485982    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.486236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.486019    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0cda92-bdf4-456b-8953-056663ce0359-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.486470 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.486447    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-certs\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.486577 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.486559    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.486636 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.486567    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-data\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.486636 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.486622    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-credential-socket\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.486813 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.486788    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0cda92-bdf4-456b-8953-056663ce0359-istiod-ca-cert\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.488286 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.488264    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-envoy\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.488782 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.488759    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0cda92-bdf4-456b-8953-056663ce0359-istio-podinfo\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.548805 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.548774    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-istio-token\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.548966 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.548942    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2s9k\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-kube-api-access-h2s9k\") pod \"e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") " pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.549697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.549683    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:13.621495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:13.619038    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:14.064702 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:03:14.064668    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d0cda92_bdf4_456b_8953_056663ce0359.slice/crio-a63a9abc8bdfffd8a90043ff19ae7b6b125b561e4324c3a7c1bf4d28a05c9d3a WatchSource:0}: Error finding container a63a9abc8bdfffd8a90043ff19ae7b6b125b561e4324c3a7c1bf4d28a05c9d3a: Status 404 returned error can't find the container with id a63a9abc8bdfffd8a90043ff19ae7b6b125b561e4324c3a7c1bf4d28a05c9d3a
Nov 03 23:03:14.067188 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.067150    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:03:14.067308 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.067234    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:03:14.067308 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.067280    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:03:14.068222 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.068199    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"]
Nov 03 23:03:14.514358 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.514321    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:14.721357 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.721319    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:14.854984 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.854900    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be81b953-ea9f-491b-be35-977a05fa2106" path="/var/lib/kubelet/pods/be81b953-ea9f-491b-be35-977a05fa2106/volumes"
Nov 03 23:03:14.914995 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.914956    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" event={"ID":"9d0cda92-bdf4-456b-8953-056663ce0359","Type":"ContainerStarted","Data":"436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50"}
Nov 03 23:03:14.915165 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:14.915045    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" event={"ID":"9d0cda92-bdf4-456b-8953-056663ce0359","Type":"ContainerStarted","Data":"a63a9abc8bdfffd8a90043ff19ae7b6b125b561e4324c3a7c1bf4d28a05c9d3a"}
Nov 03 23:03:15.161243 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.161151    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podStartSLOduration=2.161138127 podStartE2EDuration="2.161138127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:03:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:03:15.107247511 +0000 UTC m=+2398.371947522" watchObservedRunningTime="2026-11-03 23:03:15.161138127 +0000 UTC m=+2398.425838137"
Nov 03 23:03:15.562414 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.559455    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:15.562414 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.560927    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 03 23:03:15.562414 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.560984    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 03 23:03:15.611500 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.610207    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:15.815111 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.815031    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:15.880947 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.880911    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"]
Nov 03 23:03:15.881146 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:15.881108    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy" containerID="cri-o://df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884" gracePeriod=30
Nov 03 23:03:16.040181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:16.040142    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:16.563416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:16.553774    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 03 23:03:16.563416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:16.553840    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 03 23:03:17.558415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:17.552772    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 03 23:03:17.558415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:17.552836    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 03 23:03:17.770234 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:17.770188    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:18.556925 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:18.556886    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 03 23:03:18.557299 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:18.557257    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 03 23:03:18.893465 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:18.893099    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:19.184008 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:19.183975    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:19.396069 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:19.396030    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:19.562485 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:19.559311    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 03 23:03:19.562485 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:19.559413    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 03 23:03:20.558415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:20.553876    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 03 23:03:20.558415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:20.553972    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 03 23:03:20.977530 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:20.977497    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:21.232272 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.232244    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 23:03:21.237610 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237583    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-credential-socket\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.237755 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237641    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-certs\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.237755 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237675    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-socket\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.237755 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237719    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-data\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.237755 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237746    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6ce9553-9209-4c0d-b8cf-484612f07209-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6ce9553-9209-4c0d-b8cf-484612f07209-istiod-ca-cert\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.238043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237780    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-podinfo\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.238043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237818    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-token\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-token\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.238043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237857    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-credential-socket" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:21.238043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237979    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-kube-api-access-6bbrz\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-kube-api-access-6bbrz\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.238279 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.238076    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-envoy\") pod \"f6ce9553-9209-4c0d-b8cf-484612f07209\" (UID: \"f6ce9553-9209-4c0d-b8cf-484612f07209\") "
Nov 03 23:03:21.238368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237978    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-certs" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:21.238368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.237994    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-data" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:21.238368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.238007    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-socket" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:21.238653 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.238631    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.238717 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.238660    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.238717 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.238675    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.238717 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.238688    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.239219 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.239195    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6ce9553-9209-4c0d-b8cf-484612f07209-istiod-ca-cert" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:21.240336 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.240291    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-token" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:21.240552 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.240532    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-podinfo" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:03:21.240552 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.240544    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-kube-api-access-6bbrz" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "kube-api-access-6bbrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:21.240894 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.240876    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-envoy" pod "f6ce9553-9209-4c0d-b8cf-484612f07209" (UID: "f6ce9553-9209-4c0d-b8cf-484612f07209"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:21.340416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.340361    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6ce9553-9209-4c0d-b8cf-484612f07209-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.340416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.340409    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.340681 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.340426    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.340681 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.340441    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bbrz\" (UniqueName: \"kubernetes.io/projected/f6ce9553-9209-4c0d-b8cf-484612f07209-kube-api-access-6bbrz\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.340681 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.340455    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6ce9553-9209-4c0d-b8cf-484612f07209-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:21.550817 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.550719    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 03 23:03:21.550817 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.550781    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 03 23:03:21.983681 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.983646    5162 generic.go:356] "Generic (PLEG): container finished" podID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerID="df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884" exitCode=0
Nov 03 23:03:21.984163 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.983697    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"
Nov 03 23:03:21.984163 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.983752    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" event={"ID":"f6ce9553-9209-4c0d-b8cf-484612f07209","Type":"ContainerDied","Data":"df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884"}
Nov 03 23:03:21.984163 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.983851    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz" event={"ID":"f6ce9553-9209-4c0d-b8cf-484612f07209","Type":"ContainerDied","Data":"bafd99c6905f0217060261e8b2e0be664143ba06912ca520a53e8d72f54d7dca"}
Nov 03 23:03:21.984163 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.983896    5162 scope.go:122] "RemoveContainer" containerID="df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884"
Nov 03 23:03:21.997803 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.997777    5162 scope.go:122] "RemoveContainer" containerID="df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884"
Nov 03 23:03:21.998191 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:03:21.998164    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884\": container with ID starting with df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884 not found: ID does not exist" containerID="df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884"
Nov 03 23:03:21.998368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:21.998334    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884"} err="failed to get container status \"df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884\": rpc error: code = NotFound desc = could not find container \"df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884\": container with ID starting with df1efb9bd5daf7cbf62e3860f3bf4ac12c6a98f31c223fae9562498e31214884 not found: ID does not exist"
Nov 03 23:03:22.172673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:22.172637    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"]
Nov 03 23:03:22.177589 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:22.177561    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz"]
Nov 03 23:03:22.554745 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:22.554718    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:22.555005 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:22.554985    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:22.555660 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:22.555640    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:22.855298 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:22.855229    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" path="/var/lib/kubelet/pods/f6ce9553-9209-4c0d-b8cf-484612f07209/volumes"
Nov 03 23:03:24.251671 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:24.251568    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:24.818492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:24.818459    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"]
Nov 03 23:03:24.832749 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:24.832721    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"]
Nov 03 23:03:24.832922 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:24.832903    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" containerID="cri-o://9241ef14a8e7b9670929b94799efe018d7cf227a0fc0c5f4d40949ac82d47e9b" gracePeriod=30
Nov 03 23:03:24.904056 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:24.903947    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:25.003938 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:25.003902    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy" containerID="cri-o://436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50" gracePeriod=30
Nov 03 23:03:25.820254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:25.820215    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:26.018668 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:26.018633    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:28.048181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:28.048150    5162 patch_prober.go:29] interesting pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:03:28.048577 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:28.048209    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:03:29.436297 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:29.436257    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:30.039617 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.039589    5162 generic.go:356] "Generic (PLEG): container finished" podID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerID="9241ef14a8e7b9670929b94799efe018d7cf227a0fc0c5f4d40949ac82d47e9b" exitCode=0
Nov 03 23:03:30.039747 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.039647    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" event={"ID":"9f45de95-189b-4eeb-9dce-1d58d1f01359","Type":"ContainerDied","Data":"9241ef14a8e7b9670929b94799efe018d7cf227a0fc0c5f4d40949ac82d47e9b"}
Nov 03 23:03:30.133323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.133303    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 23:03:30.235498 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.235475    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:30.249854 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.249831    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-data\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.249974 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.249932    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-certs\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.249974 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.249970    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-envoy\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.250063 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.249997    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-credential-socket\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.250063 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250033    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-token\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-token\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.250169 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250087    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-kube-api-access-lghkd\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-kube-api-access-lghkd\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.250169 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250096    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-data" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.250372 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250243    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-socket\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.250372 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250287    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-certs" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.250372 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250328    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-credential-socket" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.250588 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250450    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-podinfo\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.250588 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250503    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f45de95-189b-4eeb-9dce-1d58d1f01359-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f45de95-189b-4eeb-9dce-1d58d1f01359-istiod-ca-cert\") pod \"9f45de95-189b-4eeb-9dce-1d58d1f01359\" (UID: \"9f45de95-189b-4eeb-9dce-1d58d1f01359\") "
Nov 03 23:03:30.250683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.250630    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-socket" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.251092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.251068    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.251092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.251094    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.251243 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.251110    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.251243 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.251124    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.251243 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.251073    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f45de95-189b-4eeb-9dce-1d58d1f01359-istiod-ca-cert" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:30.252458 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.252433    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-kube-api-access-lghkd" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "kube-api-access-lghkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:30.252554 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.252463    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-token" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:30.252631 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.252616    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-envoy" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.252722 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.252704    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-podinfo" pod "9f45de95-189b-4eeb-9dce-1d58d1f01359" (UID: "9f45de95-189b-4eeb-9dce-1d58d1f01359"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:03:30.352458 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352421    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-socket\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352626 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352528    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-credential-socket\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352626 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352583    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d0cda92-bdf4-456b-8953-056663ce0359-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0cda92-bdf4-456b-8953-056663ce0359-istiod-ca-cert\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352626 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352596    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-socket" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.352749 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352663    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-certs\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352749 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352701    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-envoy\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352749 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352724    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-data\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352871 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352831    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-kube-api-access-h2s9k\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-kube-api-access-h2s9k\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352871 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352854    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-credential-socket" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.352943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352883    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d0cda92-bdf4-456b-8953-056663ce0359-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0cda92-bdf4-456b-8953-056663ce0359-istio-podinfo\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.352943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352912    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-istio-token\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-istio-token\") pod \"9d0cda92-bdf4-456b-8953-056663ce0359\" (UID: \"9d0cda92-bdf4-456b-8953-056663ce0359\") "
Nov 03 23:03:30.353026 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352950    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-certs" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.353026 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.352960    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d0cda92-bdf4-456b-8953-056663ce0359-istiod-ca-cert" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:30.353134 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353049    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-data" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.353506 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353484    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353511    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353535    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353549    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0cda92-bdf4-456b-8953-056663ce0359-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353564    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lghkd\" (UniqueName: \"kubernetes.io/projected/9f45de95-189b-4eeb-9dce-1d58d1f01359-kube-api-access-lghkd\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353577    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353591    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353605    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f45de95-189b-4eeb-9dce-1d58d1f01359-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.353622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353619    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f45de95-189b-4eeb-9dce-1d58d1f01359-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.354057 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.353633    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.354988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.354963    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d0cda92-bdf4-456b-8953-056663ce0359-istio-podinfo" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:03:30.355101 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.355026    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-envoy" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:03:30.355101 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.355063    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-istio-token" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:30.355101 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.355088    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-kube-api-access-h2s9k" pod "9d0cda92-bdf4-456b-8953-056663ce0359" (UID: "9d0cda92-bdf4-456b-8953-056663ce0359"). InnerVolumeSpecName "kube-api-access-h2s9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:30.455127 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.455101    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0cda92-bdf4-456b-8953-056663ce0359-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.455490 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.455131    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2s9k\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-kube-api-access-h2s9k\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.455490 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.455145    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0cda92-bdf4-456b-8953-056663ce0359-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:30.455490 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:30.455158    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0cda92-bdf4-456b-8953-056663ce0359-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:03:31.047754 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.047718    5162 generic.go:356] "Generic (PLEG): container finished" podID="9d0cda92-bdf4-456b-8953-056663ce0359" containerID="436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50" exitCode=0
Nov 03 23:03:31.047926 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.047782    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"
Nov 03 23:03:31.047926 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.047798    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" event={"ID":"9d0cda92-bdf4-456b-8953-056663ce0359","Type":"ContainerDied","Data":"436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50"}
Nov 03 23:03:31.047926 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.047866    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6" event={"ID":"9d0cda92-bdf4-456b-8953-056663ce0359","Type":"ContainerDied","Data":"a63a9abc8bdfffd8a90043ff19ae7b6b125b561e4324c3a7c1bf4d28a05c9d3a"}
Nov 03 23:03:31.047926 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.047897    5162 scope.go:122] "RemoveContainer" containerID="436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50"
Nov 03 23:03:31.049515 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.049489    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl" event={"ID":"9f45de95-189b-4eeb-9dce-1d58d1f01359","Type":"ContainerDied","Data":"8457e0803a4fe4490c1fb42c99042f6161671f5d3537c4a87621ff713e67c1ee"}
Nov 03 23:03:31.049648 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.049580    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"
Nov 03 23:03:31.059538 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.059521    5162 scope.go:122] "RemoveContainer" containerID="436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50"
Nov 03 23:03:31.059823 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:03:31.059805    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50\": container with ID starting with 436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50 not found: ID does not exist" containerID="436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50"
Nov 03 23:03:31.059879 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.059831    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50"} err="failed to get container status \"436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50\": rpc error: code = NotFound desc = could not find container \"436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50\": container with ID starting with 436e50e0b06fe0bc9b3a71795524d59f3f742b3bd83bade8c09d93ee2a372d50 not found: ID does not exist"
Nov 03 23:03:31.059879 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.059848    5162 scope.go:122] "RemoveContainer" containerID="9241ef14a8e7b9670929b94799efe018d7cf227a0fc0c5f4d40949ac82d47e9b"
Nov 03 23:03:31.239330 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.239298    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"]
Nov 03 23:03:31.242511 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.242486    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6"]
Nov 03 23:03:31.355790 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.355749    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:31.365066 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.365029    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"]
Nov 03 23:03:31.368159 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.368137    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl"]
Nov 03 23:03:31.584715 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:31.584628    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:32.854593 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:32.854563    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" path="/var/lib/kubelet/pods/9d0cda92-bdf4-456b-8953-056663ce0359/volumes"
Nov 03 23:03:32.855057 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:32.855042    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" path="/var/lib/kubelet/pods/9f45de95-189b-4eeb-9dce-1d58d1f01359/volumes"
Nov 03 23:03:34.302480 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:34.302445    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:34.934417 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:34.932586    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:35.134141 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:35.134102    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:36.108089 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:36.108052    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:39.038695 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:39.038660    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:40.082677 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:03:40.082642    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:03:40.750456 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:40.750423    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:43.966188 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:43.966156    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:47.049805 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:47.049767    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:48.817236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:48.817198    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:52.747258 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:52.747219    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:52.948099 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:52.948067    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:53.294041 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.294009    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"]
Nov 03 23:03:53.294809 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.294791    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy"
Nov 03 23:03:53.294882 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.294813    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy"
Nov 03 23:03:53.294882 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.294839    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy"
Nov 03 23:03:53.294882 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.294848    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy"
Nov 03 23:03:53.294997 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.294882    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy"
Nov 03 23:03:53.294997 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.294891    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy"
Nov 03 23:03:53.295076 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.295015    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f45de95-189b-4eeb-9dce-1d58d1f01359" containerName="istio-proxy"
Nov 03 23:03:53.295076 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.295034    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6ce9553-9209-4c0d-b8cf-484612f07209" containerName="istio-proxy"
Nov 03 23:03:53.295076 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.295050    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d0cda92-bdf4-456b-8953-056663ce0359" containerName="istio-proxy"
Nov 03 23:03:53.298854 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.298833    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.351922 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.351893    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"]
Nov 03 23:03:53.400106 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.400073    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ac6633-serving-cert\""
Nov 03 23:03:53.431578 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.431545    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd9e7fd-4f46-48ba-9947-75a14210c989-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.431765 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.431719    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-metrics-certs\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.431876 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.431854    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzppr\" (UniqueName: \"kubernetes.io/projected/abd9e7fd-4f46-48ba-9947-75a14210c989-kube-api-access-zzppr\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.432026 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.432009    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.547340 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.547238    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-metrics-certs\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.548733 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.548696    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzppr\" (UniqueName: \"kubernetes.io/projected/abd9e7fd-4f46-48ba-9947-75a14210c989-kube-api-access-zzppr\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.549133 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.549116    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.552277 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.552255    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd9e7fd-4f46-48ba-9947-75a14210c989-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.552853 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.552051    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:53.552853 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:03:53.550104    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-serving-cert: secret "maas-api-e2e-inf-b-ac6633-serving-cert" not found
Nov 03 23:03:53.552853 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:03:53.552639    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls podName:abd9e7fd-4f46-48ba-9947-75a14210c989 nodeName:}" failed. No retries permitted until 2026-11-03 23:03:54.052615322 +0000 UTC m=+2437.317315330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls") pod "maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x" (UID: "abd9e7fd-4f46-48ba-9947-75a14210c989") : secret "maas-api-e2e-inf-b-ac6633-serving-cert" not found
Nov 03 23:03:53.553449 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.553325    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd9e7fd-4f46-48ba-9947-75a14210c989-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.558574 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.558553    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-metrics-certs\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.620932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.620900    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzppr\" (UniqueName: \"kubernetes.io/projected/abd9e7fd-4f46-48ba-9947-75a14210c989-kube-api-access-zzppr\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:53.762789 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.762699    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:53.970287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:53.970256    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:03:54.059223 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:54.059194    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:54.061633 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:54.061608    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls\") pod \"maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:54.212969 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:54.212941    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:54.586749 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:03:54.586715    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd9e7fd_4f46_48ba_9947_75a14210c989.slice/crio-5b4ed8502438f5f7f1221e59bfb72bbfd7c4b4396a83d7f70d1bf96194d4c73e WatchSource:0}: Error finding container 5b4ed8502438f5f7f1221e59bfb72bbfd7c4b4396a83d7f70d1bf96194d4c73e: Status 404 returned error can't find the container with id 5b4ed8502438f5f7f1221e59bfb72bbfd7c4b4396a83d7f70d1bf96194d4c73e
Nov 03 23:03:54.588596 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:54.588577    5162 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 23:03:54.589682 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:54.589662    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"]
Nov 03 23:03:55.202263 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:55.202227    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x" event={"ID":"abd9e7fd-4f46-48ba-9947-75a14210c989","Type":"ContainerStarted","Data":"5b4ed8502438f5f7f1221e59bfb72bbfd7c4b4396a83d7f70d1bf96194d4c73e"}
Nov 03 23:03:56.213641 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:56.213600    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x" event={"ID":"abd9e7fd-4f46-48ba-9947-75a14210c989","Type":"ContainerStarted","Data":"3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08"}
Nov 03 23:03:56.214043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:56.213679    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:03:58.527669 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:03:58.527624    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:02.224353 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:02.224323    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:04:02.348279 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:02.348222    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x" podStartSLOduration=8.446093063 podStartE2EDuration="9.348209711s" totalImagesPullingTime="902.116648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:03:56.342076346 +0000 UTC m=+2439.606776359" watchObservedRunningTime="2026-11-03 23:04:02.348209711 +0000 UTC m=+2445.612909721"
Nov 03 23:04:03.332797 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:03.332759    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:07.034797 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:07.034763    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:08.135074 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:08.135037    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:10.083405 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:04:10.083361    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:04:12.924421 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:12.924301    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:17.118692 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:17.118651    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:18.638638 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:18.638596    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:18.955467 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:18.955333    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:19.239827 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:19.239742    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:21.345971 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:21.345941    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:24.019133 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.019093    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:24.411509 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.411480    5162 generic.go:356] "Generic (PLEG): container finished" podID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerID="2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55" exitCode=0
Nov 03 23:04:24.411657 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.411557    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" event={"ID":"013e4f6a-2cf6-4d66-8580-77c5e2359f53","Type":"ContainerDied","Data":"2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55"}
Nov 03 23:04:24.412002 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.411985    5162 scope.go:122] "RemoveContainer" containerID="2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55"
Nov 03 23:04:24.413435 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.413377    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:04:24.413529 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.413474    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:04:24.413529 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.413509    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:04:24.873190 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:24.873161    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:04:25.420090 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:25.420056    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" event={"ID":"013e4f6a-2cf6-4d66-8580-77c5e2359f53","Type":"ContainerStarted","Data":"6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806"}
Nov 03 23:04:26.163804 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:26.163770    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:04:26.165558 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:26.165526    5162 patch_prober.go:29] interesting pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 03 23:04:26.165723 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:26.165578    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 03 23:04:27.161772 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:27.161732    5162 patch_prober.go:29] interesting pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 03 23:04:27.162375 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:27.162339    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 03 23:04:28.163822 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:28.163783    5162 patch_prober.go:29] interesting pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 03 23:04:28.164458 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:28.164424    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 03 23:04:28.829845 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:28.829800    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:29.029514 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:29.029474    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:29.162816 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:29.162730    5162 patch_prober.go:29] interesting pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 03 23:04:29.163058 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:29.163029    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 03 23:04:29.578217 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:29.578185    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:30.165562 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:30.165517    5162 patch_prober.go:29] interesting pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 03 23:04:30.165760 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:30.165580    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 03 23:04:30.974624 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:30.974590    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:31.202856 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:31.202820    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:04:31.203615 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:31.203596    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:04:31.204375 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:31.204356    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:04:33.070623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:33.070583    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:34.671642 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:34.671601    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:34.924821 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:34.924738    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:35.493380 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:35.493343    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:36.082199 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:36.082160    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:36.287848 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:36.287812    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:36.390500 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:36.390368    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"]
Nov 03 23:04:36.391058 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:36.391001    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy" containerID="cri-o://d817f356aa9448197e72934311f2662c932f23f656d8eb9c9b7d323d82e6db9d" gracePeriod=30
Nov 03 23:04:36.493414 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:36.492787    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:40.015430 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:40.015374    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:40.084499 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:04:40.084463    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:04:41.538692 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.538642    5162 generic.go:356] "Generic (PLEG): container finished" podID="b9233afc-7975-4d25-b869-612a49d4038a" containerID="d817f356aa9448197e72934311f2662c932f23f656d8eb9c9b7d323d82e6db9d" exitCode=0
Nov 03 23:04:41.539181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.538699    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" event={"ID":"b9233afc-7975-4d25-b869-612a49d4038a","Type":"ContainerDied","Data":"d817f356aa9448197e72934311f2662c932f23f656d8eb9c9b7d323d82e6db9d"}
Nov 03 23:04:41.539181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.538761    5162 scope.go:122] "RemoveContainer" containerID="2d68b272f480080d36b2359bf1cb3dc9afda3f76fc689b54431ab4d205f1ff65"
Nov 03 23:04:41.665473 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.665140    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 23:04:41.759595 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759564    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-data\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.759743 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759613    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-istio-token\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-istio-token\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.759743 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759651    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9233afc-7975-4d25-b869-612a49d4038a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9233afc-7975-4d25-b869-612a49d4038a-istiod-ca-cert\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.759743 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759701    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-kube-api-access-clnn7\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-kube-api-access-clnn7\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.759908 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759813    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-socket\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.759908 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759808    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-data" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:41.760016 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759908    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-certs\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.760016 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759942    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b9233afc-7975-4d25-b869-612a49d4038a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9233afc-7975-4d25-b869-612a49d4038a-istio-podinfo\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.760016 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.759981    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-envoy\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.760167 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760022    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-credential-socket\") pod \"b9233afc-7975-4d25-b869-612a49d4038a\" (UID: \"b9233afc-7975-4d25-b869-612a49d4038a\") "
Nov 03 23:04:41.760167 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760060    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-socket" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:41.760167 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760118    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-certs" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:41.760327 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760162    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9233afc-7975-4d25-b869-612a49d4038a-istiod-ca-cert" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:04:41.760424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760384    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-credential-socket" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:41.760616 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760599    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.760692 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760620    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.760692 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760635    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.760692 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760649    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.760692 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.760662    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9233afc-7975-4d25-b869-612a49d4038a-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.761950 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.761924    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-istio-token" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:41.762072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.761967    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9233afc-7975-4d25-b869-612a49d4038a-istio-podinfo" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:04:41.762246 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.762229    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-kube-api-access-clnn7" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "kube-api-access-clnn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:41.762311 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.762284    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-envoy" pod "b9233afc-7975-4d25-b869-612a49d4038a" (UID: "b9233afc-7975-4d25-b869-612a49d4038a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:41.863070 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.862862    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.863070 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.863013    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clnn7\" (UniqueName: \"kubernetes.io/projected/b9233afc-7975-4d25-b869-612a49d4038a-kube-api-access-clnn7\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.863070 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.863039    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9233afc-7975-4d25-b869-612a49d4038a-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:41.863070 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:41.863057    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9233afc-7975-4d25-b869-612a49d4038a-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:04:42.268106 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.268069    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:42.546175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.546095    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"
Nov 03 23:04:42.546175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.546128    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b" event={"ID":"b9233afc-7975-4d25-b869-612a49d4038a","Type":"ContainerDied","Data":"73d5dd67356775109a403fe52b4064d2af03e41fbd1e1bb4a6fc6697b7746998"}
Nov 03 23:04:42.546650 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.546190    5162 scope.go:122] "RemoveContainer" containerID="d817f356aa9448197e72934311f2662c932f23f656d8eb9c9b7d323d82e6db9d"
Nov 03 23:04:42.553419 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.553360    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:42.769306 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.769270    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"]
Nov 03 23:04:42.775533 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.775505    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:42.821026 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.820962    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b"]
Nov 03 23:04:42.854061 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:42.854029    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9233afc-7975-4d25-b869-612a49d4038a" path="/var/lib/kubelet/pods/b9233afc-7975-4d25-b869-612a49d4038a/volumes"
Nov 03 23:04:45.315812 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:45.315771    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:48.840385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:48.840350    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:50.459322 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:50.457054    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:51.721924 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:51.721226    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:55.384131 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:55.384092    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:55.997870 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:55.997831    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:04:56.214653 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:04:56.214615    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:00.114220 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:00.114178    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:05.009617 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:05.009521    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:07.359759 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:07.359723    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:09.821408 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:09.821321    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:10.083703 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:05:10.083616    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:05:14.718198 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:14.718144    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:19.622885 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:19.622852    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:21.673413 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:21.673356    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:24.418346 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:24.418312    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:29.212618 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:29.212581    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:31.316638 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:31.316600    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:34.018932 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:34.018896    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:38.833325 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:38.833242    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:40.083463 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:05:40.083428    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:05:40.685280 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:40.685242    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:43.602649 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:43.602598    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:48.417435 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:48.417378    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:53.320595 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:53.320549    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:05:58.151839 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:05:58.151798    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:02.316365 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:02.316334    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:03.897682 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:03.897640    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:04.108686 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:04.108634    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:04.313474 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:04.313439    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:04.799013 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:04.798963    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:05.049480 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:05.049371    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:07.702658 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:07.702625    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:09.221087 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:09.221051    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:10.084150 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:06:10.084115    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:06:11.429430 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:11.429380    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:11.687553 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:11.687475    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:11.901606 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:11.901577    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:12.289258 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:12.289229    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:14.360153 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:14.360117    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:17.183941 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:17.183908    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"]
Nov 03 23:06:17.395124 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:17.395083    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:19.532240 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:19.532206    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:22.030903 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:22.030858    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:24.420770 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:24.420735    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:24.716828 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:24.716742    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:24.919830 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:24.919801    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:29.213610 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:29.213559    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:30.282689 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:30.282583    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:31.679053 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:31.678992    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:32.204360 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.204308    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main" containerID="cri-o://c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3" gracePeriod=45
Nov 03 23:06:32.464863 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.464809    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 23:06:32.573303 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573278    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-model-cache\") pod \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") "
Nov 03 23:06:32.573481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573328    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-dshm\") pod \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") "
Nov 03 23:06:32.573481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573350    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tmp-dir\") pod \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") "
Nov 03 23:06:32.573481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573379    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-kube-api-access-m4hqg\" (UniqueName: \"kubernetes.io/projected/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-kube-api-access-m4hqg\") pod \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") "
Nov 03 23:06:32.573481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573438    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-home\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-home\") pod \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") "
Nov 03 23:06:32.573695 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573567    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-model-cache" pod "2d28165a-2bf2-44b9-be71-d3cc7ed742e5" (UID: "2d28165a-2bf2-44b9-be71-d3cc7ed742e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:32.573695 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573640    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tls-certs\") pod \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\" (UID: \"2d28165a-2bf2-44b9-be71-d3cc7ed742e5\") "
Nov 03 23:06:32.573695 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573645    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tmp-dir" pod "2d28165a-2bf2-44b9-be71-d3cc7ed742e5" (UID: "2d28165a-2bf2-44b9-be71-d3cc7ed742e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:32.573840 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.573743    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-home" pod "2d28165a-2bf2-44b9-be71-d3cc7ed742e5" (UID: "2d28165a-2bf2-44b9-be71-d3cc7ed742e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:32.574065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.574045    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:06:32.574118 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.574095    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:06:32.574118 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.574111    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:06:32.575597 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.575577    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-dshm" pod "2d28165a-2bf2-44b9-be71-d3cc7ed742e5" (UID: "2d28165a-2bf2-44b9-be71-d3cc7ed742e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:32.575676 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.575589    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tls-certs" pod "2d28165a-2bf2-44b9-be71-d3cc7ed742e5" (UID: "2d28165a-2bf2-44b9-be71-d3cc7ed742e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:06:32.575676 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.575611    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-kube-api-access-m4hqg" pod "2d28165a-2bf2-44b9-be71-d3cc7ed742e5" (UID: "2d28165a-2bf2-44b9-be71-d3cc7ed742e5"). InnerVolumeSpecName "kube-api-access-m4hqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:32.675348 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.675313    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:06:32.675348 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.675348    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:06:32.675558 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:32.675365    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4hqg\" (UniqueName: \"kubernetes.io/projected/2d28165a-2bf2-44b9-be71-d3cc7ed742e5-kube-api-access-m4hqg\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:06:33.240511 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.240475    5162 generic.go:356] "Generic (PLEG): container finished" podID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerID="c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3" exitCode=0
Nov 03 23:06:33.240999 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.240526    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"
Nov 03 23:06:33.240999 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.240562    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" event={"ID":"2d28165a-2bf2-44b9-be71-d3cc7ed742e5","Type":"ContainerDied","Data":"c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3"}
Nov 03 23:06:33.240999 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.240604    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48" event={"ID":"2d28165a-2bf2-44b9-be71-d3cc7ed742e5","Type":"ContainerDied","Data":"956bf034f62ce587af4d8a8bcda83105c29655aaf856231cba9601248fd31704"}
Nov 03 23:06:33.240999 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.240622    5162 scope.go:122] "RemoveContainer" containerID="c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3"
Nov 03 23:06:33.249685 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.249663    5162 scope.go:122] "RemoveContainer" containerID="c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3"
Nov 03 23:06:33.249957 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:06:33.249937    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3\": container with ID starting with c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3 not found: ID does not exist" containerID="c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3"
Nov 03 23:06:33.250004 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.249967    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3"} err="failed to get container status \"c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3\": rpc error: code = NotFound desc = could not find container \"c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3\": container with ID starting with c6fa9615a7a8b0ff8b21f33430fad61e867227bfd0d52a037e171273b6b244b3 not found: ID does not exist"
Nov 03 23:06:33.411874 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.411845    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"]
Nov 03 23:06:33.463176 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:33.463149    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-5e72ad/distinct-2-w6-5e72ad-kserve-c54bf4d86-q7z48"]
Nov 03 23:06:34.463358 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:34.463318    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:34.854431 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:34.854340    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" path="/var/lib/kubelet/pods/2d28165a-2bf2-44b9-be71-d3cc7ed742e5/volumes"
Nov 03 23:06:39.306901 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:39.306870    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:40.032836 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:40.032794    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:40.083658 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:06:40.083628    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:06:40.283894 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:40.283810    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:44.126342 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:44.124772    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:45.060220 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:45.060184    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:45.279373 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:45.279342    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:48.711491 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:48.711452    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:50.179772 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:50.179729    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:50.427823 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:50.427793    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:50.682741 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:50.682713    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:51.113927 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:51.113848    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:54.897443 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:54.897375    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:55.252852 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:55.252807    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:55.457416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:55.457355    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:55.660323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:55.660238    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:57.270729 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:57.270697    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:57.473748 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:57.473707    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:06:57.677631 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:06:57.677599    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:00.718988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:00.718947    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:01.283795 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:01.283762    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:05.626907 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:05.626854    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:08.067581 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:08.067491    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:10.084704 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:07:10.084650    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:07:10.308929 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:10.308386    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:10.517442 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:10.517376    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:15.444170 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:15.444138    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:16.430372 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:16.428279    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:16.727492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:16.727363    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:19.583333 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.583293    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"]
Nov 03 23:07:19.583806 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.583500    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh" podUID="46dcc5d8-b2d9-4c60-9e76-8cf25739094b" containerName="maas-api" containerID="cri-o://9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897" gracePeriod=30
Nov 03 23:07:19.731746 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.731714    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:19.859083 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.859055    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 23:07:19.972235 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.972192    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls\") pod \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") "
Nov 03 23:07:19.973052 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.972995    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-kube-api-access-59b8b\" (UniqueName: \"kubernetes.io/projected/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-kube-api-access-59b8b\") pod \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") "
Nov 03 23:07:19.985832 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.985803    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-metrics-certs\") pod \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") "
Nov 03 23:07:19.986194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.986175    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-openshift-service-ca\") pod \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\" (UID: \"46dcc5d8-b2d9-4c60-9e76-8cf25739094b\") "
Nov 03 23:07:19.988809 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.988770    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-openshift-service-ca" pod "46dcc5d8-b2d9-4c60-9e76-8cf25739094b" (UID: "46dcc5d8-b2d9-4c60-9e76-8cf25739094b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:19.996790 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.995507    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls" pod "46dcc5d8-b2d9-4c60-9e76-8cf25739094b" (UID: "46dcc5d8-b2d9-4c60-9e76-8cf25739094b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:19.999002 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:19.998963    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-kube-api-access-59b8b" pod "46dcc5d8-b2d9-4c60-9e76-8cf25739094b" (UID: "46dcc5d8-b2d9-4c60-9e76-8cf25739094b"). InnerVolumeSpecName "kube-api-access-59b8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:20.012542 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.012506    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-metrics-certs" pod "46dcc5d8-b2d9-4c60-9e76-8cf25739094b" (UID: "46dcc5d8-b2d9-4c60-9e76-8cf25739094b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:20.088784 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.088747    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59b8b\" (UniqueName: \"kubernetes.io/projected/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-kube-api-access-59b8b\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:20.089001 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.088963    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:20.089125 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.089108    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:20.089221 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.089208    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46dcc5d8-b2d9-4c60-9e76-8cf25739094b-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:20.536728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.529276    5162 generic.go:356] "Generic (PLEG): container finished" podID="46dcc5d8-b2d9-4c60-9e76-8cf25739094b" containerID="9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897" exitCode=0
Nov 03 23:07:20.536728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.529485    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh" event={"ID":"46dcc5d8-b2d9-4c60-9e76-8cf25739094b","Type":"ContainerDied","Data":"9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897"}
Nov 03 23:07:20.536728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.529539    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh" event={"ID":"46dcc5d8-b2d9-4c60-9e76-8cf25739094b","Type":"ContainerDied","Data":"c5f1ef844710c4fc7dad768dd772a2c6c4ec945d93430f1c1199378c5ea367fc"}
Nov 03 23:07:20.536728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.529566    5162 scope.go:122] "RemoveContainer" containerID="9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897"
Nov 03 23:07:20.536728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.532277    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"
Nov 03 23:07:20.558914 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.558879    5162 scope.go:122] "RemoveContainer" containerID="9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897"
Nov 03 23:07:20.559347 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:07:20.559314    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897\": container with ID starting with 9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897 not found: ID does not exist" containerID="9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897"
Nov 03 23:07:20.559454 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.559358    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897"} err="failed to get container status \"9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897\": rpc error: code = NotFound desc = could not find container \"9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897\": container with ID starting with 9699135f33c3c71e7b08b2b9410cdaab4eb4ac3d0b81cf53e9f9bfd38d1d0897 not found: ID does not exist"
Nov 03 23:07:20.706366 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.706331    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"]
Nov 03 23:07:20.710192 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.710170    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-5e72ad-668fcd6d6f-vc4gh"]
Nov 03 23:07:20.854153 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.854074    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46dcc5d8-b2d9-4c60-9e76-8cf25739094b" path="/var/lib/kubelet/pods/46dcc5d8-b2d9-4c60-9e76-8cf25739094b/volumes"
Nov 03 23:07:20.889346 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:20.889306    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:21.275412 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:21.275362    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:21.551797 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:21.551715    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:22.187735 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:22.187697    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:22.396408 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:22.396365    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:25.454708 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:25.454671    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:27.833260 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:27.833227    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:28.129648 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:28.129561    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:28.339881 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:28.339844    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:29.635233 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:29.634211    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:29.892859 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:29.892776    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:30.090084 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:30.090045    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:30.457317 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:30.457288    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:32.039560 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:32.039526    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:32.946117 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:32.945911    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:35.359508 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:35.359477    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"]
Nov 03 23:07:35.411206 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:35.411177    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"]
Nov 03 23:07:35.413753 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:35.413725    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"]
Nov 03 23:07:35.567634 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:35.567601    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:38.126265 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:38.126215    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:38.583913 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:38.583883    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:40.083081 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:07:40.083043    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:07:42.849309 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:42.849272    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:43.059851 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:43.059816    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:43.315466 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:43.315430    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:48.143335 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:48.143296    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:50.379742 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.379707    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main" containerID="cri-o://ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192" gracePeriod=45
Nov 03 23:07:50.430743 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.430709    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main" containerID="cri-o://89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb" gracePeriod=45
Nov 03 23:07:50.432919 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.432891    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" containerID="cri-o://e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d" gracePeriod=45
Nov 03 23:07:50.701379 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.701353    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 23:07:50.704515 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.704495    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 23:07:50.720623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.720603    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 23:07:50.722123 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.722097    5162 generic.go:356] "Generic (PLEG): container finished" podID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerID="ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192" exitCode=0
Nov 03 23:07:50.722236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.722143    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"
Nov 03 23:07:50.722236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.722179    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" event={"ID":"59552c85-6ea7-42f0-9b94-eb639f8f0dfd","Type":"ContainerDied","Data":"ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192"}
Nov 03 23:07:50.722236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.722219    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk" event={"ID":"59552c85-6ea7-42f0-9b94-eb639f8f0dfd","Type":"ContainerDied","Data":"11c6cfa60f9ebca47433c5ff6ae86a6c46bc9d6301a4e42a36fe5cbf8b81f8e3"}
Nov 03 23:07:50.722384 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.722239    5162 scope.go:122] "RemoveContainer" containerID="ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192"
Nov 03 23:07:50.723665 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.723642    5162 generic.go:356] "Generic (PLEG): container finished" podID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerID="e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d" exitCode=0
Nov 03 23:07:50.723869 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.723687    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"
Nov 03 23:07:50.723978 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.723703    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" event={"ID":"ede9ac93-5f42-48c8-85a0-848d53fa5056","Type":"ContainerDied","Data":"e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d"}
Nov 03 23:07:50.724031 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.724013    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" event={"ID":"ede9ac93-5f42-48c8-85a0-848d53fa5056","Type":"ContainerDied","Data":"d8940d8f01005d3b05c18c139dde9182de9c4ced48f86311758624d7b14de830"}
Nov 03 23:07:50.725362 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.725338    5162 generic.go:356] "Generic (PLEG): container finished" podID="c73f4e32-0faa-48a1-8953-eca91366445c" containerID="89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb" exitCode=0
Nov 03 23:07:50.725482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.725368    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"
Nov 03 23:07:50.725482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.725423    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" event={"ID":"c73f4e32-0faa-48a1-8953-eca91366445c","Type":"ContainerDied","Data":"89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb"}
Nov 03 23:07:50.725482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.725452    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j" event={"ID":"c73f4e32-0faa-48a1-8953-eca91366445c","Type":"ContainerDied","Data":"e6703fc4c270be4f10603e74d7672799762bf5ddbe88de0c54149474cbe56d71"}
Nov 03 23:07:50.731262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.731245    5162 scope.go:122] "RemoveContainer" containerID="ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192"
Nov 03 23:07:50.731558 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:07:50.731532    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192\": container with ID starting with ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192 not found: ID does not exist" containerID="ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192"
Nov 03 23:07:50.731615 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.731559    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192"} err="failed to get container status \"ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192\": rpc error: code = NotFound desc = could not find container \"ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192\": container with ID starting with ac08dec485312215071475f034101ea90562b9153465fb3e6d61b5f170275192 not found: ID does not exist"
Nov 03 23:07:50.731615 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.731575    5162 scope.go:122] "RemoveContainer" containerID="e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d"
Nov 03 23:07:50.739337 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.739321    5162 scope.go:122] "RemoveContainer" containerID="e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d"
Nov 03 23:07:50.739697 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:07:50.739680    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d\": container with ID starting with e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d not found: ID does not exist" containerID="e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d"
Nov 03 23:07:50.739758 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.739709    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d"} err="failed to get container status \"e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d\": rpc error: code = NotFound desc = could not find container \"e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d\": container with ID starting with e0fa9db299e30637ec821f67c1601410a07af895ef4991940f1ae9b07a06d34d not found: ID does not exist"
Nov 03 23:07:50.739758 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.739725    5162 scope.go:122] "RemoveContainer" containerID="89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb"
Nov 03 23:07:50.749825 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.749803    5162 scope.go:122] "RemoveContainer" containerID="89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb"
Nov 03 23:07:50.750231 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:07:50.750212    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb\": container with ID starting with 89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb not found: ID does not exist" containerID="89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb"
Nov 03 23:07:50.750320 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.750235    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb"} err="failed to get container status \"89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb\": rpc error: code = NotFound desc = could not find container \"89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb\": container with ID starting with 89524c6b99759f1e05ec70537dcbb35b96c2c738dff2a65559a256a544d3fdcb not found: ID does not exist"
Nov 03 23:07:50.850998 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.850971    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tmp-dir\") pod \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") "
Nov 03 23:07:50.851181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851010    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-dshm\") pod \"c73f4e32-0faa-48a1-8953-eca91366445c\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") "
Nov 03 23:07:50.851181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851037    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-model-cache\") pod \"ede9ac93-5f42-48c8-85a0-848d53fa5056\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") "
Nov 03 23:07:50.851181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851061    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-home\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-home\") pod \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") "
Nov 03 23:07:50.851181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851087    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ede9ac93-5f42-48c8-85a0-848d53fa5056-tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9ac93-5f42-48c8-85a0-848d53fa5056-tls-certs\") pod \"ede9ac93-5f42-48c8-85a0-848d53fa5056\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") "
Nov 03 23:07:50.851181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851139    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tls-certs\" (UniqueName: \"kubernetes.io/secret/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tls-certs\") pod \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") "
Nov 03 23:07:50.851181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851167    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-home\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-home\") pod \"c73f4e32-0faa-48a1-8953-eca91366445c\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851199    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-dshm\") pod \"ede9ac93-5f42-48c8-85a0-848d53fa5056\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851260    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c73f4e32-0faa-48a1-8953-eca91366445c-kube-api-access-v9p4t\" (UniqueName: \"kubernetes.io/projected/c73f4e32-0faa-48a1-8953-eca91366445c-kube-api-access-v9p4t\") pod \"c73f4e32-0faa-48a1-8953-eca91366445c\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851292    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tmp-dir" pod "59552c85-6ea7-42f0-9b94-eb639f8f0dfd" (UID: "59552c85-6ea7-42f0-9b94-eb639f8f0dfd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851322    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c73f4e32-0faa-48a1-8953-eca91366445c-tls-certs\" (UniqueName: \"kubernetes.io/secret/c73f4e32-0faa-48a1-8953-eca91366445c-tls-certs\") pod \"c73f4e32-0faa-48a1-8953-eca91366445c\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851320    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-model-cache" pod "ede9ac93-5f42-48c8-85a0-848d53fa5056" (UID: "ede9ac93-5f42-48c8-85a0-848d53fa5056"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851355    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-home\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-home\") pod \"ede9ac93-5f42-48c8-85a0-848d53fa5056\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851379    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-home" pod "59552c85-6ea7-42f0-9b94-eb639f8f0dfd" (UID: "59552c85-6ea7-42f0-9b94-eb639f8f0dfd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851422    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-tmp-dir\") pod \"c73f4e32-0faa-48a1-8953-eca91366445c\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851463    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-model-cache\") pod \"c73f4e32-0faa-48a1-8953-eca91366445c\" (UID: \"c73f4e32-0faa-48a1-8953-eca91366445c\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851500    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-dshm\") pod \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") "
Nov 03 23:07:50.851568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851568    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ede9ac93-5f42-48c8-85a0-848d53fa5056-kube-api-access-kp7jd\" (UniqueName: \"kubernetes.io/projected/ede9ac93-5f42-48c8-85a0-848d53fa5056-kube-api-access-kp7jd\") pod \"ede9ac93-5f42-48c8-85a0-848d53fa5056\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") "
Nov 03 23:07:50.852135 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851600    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-model-cache\") pod \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") "
Nov 03 23:07:50.852135 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851621    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-home" pod "c73f4e32-0faa-48a1-8953-eca91366445c" (UID: "c73f4e32-0faa-48a1-8953-eca91366445c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.852135 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851651    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-kube-api-access-zwkrc\") pod \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\" (UID: \"59552c85-6ea7-42f0-9b94-eb639f8f0dfd\") "
Nov 03 23:07:50.852135 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851701    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-tmp-dir\") pod \"ede9ac93-5f42-48c8-85a0-848d53fa5056\" (UID: \"ede9ac93-5f42-48c8-85a0-848d53fa5056\") "
Nov 03 23:07:50.852135 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.851778    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-tmp-dir" pod "c73f4e32-0faa-48a1-8953-eca91366445c" (UID: "c73f4e32-0faa-48a1-8953-eca91366445c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.852432 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.852268    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-model-cache" pod "59552c85-6ea7-42f0-9b94-eb639f8f0dfd" (UID: "59552c85-6ea7-42f0-9b94-eb639f8f0dfd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.852432 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.852317    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-home" pod "ede9ac93-5f42-48c8-85a0-848d53fa5056" (UID: "ede9ac93-5f42-48c8-85a0-848d53fa5056"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.852615 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.852527    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-model-cache" pod "c73f4e32-0faa-48a1-8953-eca91366445c" (UID: "c73f4e32-0faa-48a1-8953-eca91366445c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.852615 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.852604    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-tmp-dir" pod "ede9ac93-5f42-48c8-85a0-848d53fa5056" (UID: "ede9ac93-5f42-48c8-85a0-848d53fa5056"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.852821 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.852802    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.852981 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.852966    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.853096 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.853077    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.853215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.853100    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.853215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.853115    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.853215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.853129    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.853215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.853143    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.853215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.853157    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.853215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.853172    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.854224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.854178    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-dshm" pod "c73f4e32-0faa-48a1-8953-eca91366445c" (UID: "c73f4e32-0faa-48a1-8953-eca91366445c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.854435 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.854373    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c73f4e32-0faa-48a1-8953-eca91366445c-tls-certs" pod "c73f4e32-0faa-48a1-8953-eca91366445c" (UID: "c73f4e32-0faa-48a1-8953-eca91366445c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:50.854546 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.854521    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-dshm" pod "ede9ac93-5f42-48c8-85a0-848d53fa5056" (UID: "ede9ac93-5f42-48c8-85a0-848d53fa5056"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.854611 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.854547    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73f4e32-0faa-48a1-8953-eca91366445c-kube-api-access-v9p4t" pod "c73f4e32-0faa-48a1-8953-eca91366445c" (UID: "c73f4e32-0faa-48a1-8953-eca91366445c"). InnerVolumeSpecName "kube-api-access-v9p4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:50.855215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.855185    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-kube-api-access-zwkrc" pod "59552c85-6ea7-42f0-9b94-eb639f8f0dfd" (UID: "59552c85-6ea7-42f0-9b94-eb639f8f0dfd"). InnerVolumeSpecName "kube-api-access-zwkrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:50.855485 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.855463    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tls-certs" pod "59552c85-6ea7-42f0-9b94-eb639f8f0dfd" (UID: "59552c85-6ea7-42f0-9b94-eb639f8f0dfd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:50.855485 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.855472    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ede9ac93-5f42-48c8-85a0-848d53fa5056-tls-certs" pod "ede9ac93-5f42-48c8-85a0-848d53fa5056" (UID: "ede9ac93-5f42-48c8-85a0-848d53fa5056"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:50.855610 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.855560    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-dshm" pod "59552c85-6ea7-42f0-9b94-eb639f8f0dfd" (UID: "59552c85-6ea7-42f0-9b94-eb639f8f0dfd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:50.856137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.856120    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede9ac93-5f42-48c8-85a0-848d53fa5056-kube-api-access-kp7jd" pod "ede9ac93-5f42-48c8-85a0-848d53fa5056" (UID: "ede9ac93-5f42-48c8-85a0-848d53fa5056"). InnerVolumeSpecName "kube-api-access-kp7jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:50.954424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954381    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73f4e32-0faa-48a1-8953-eca91366445c-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954422    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ede9ac93-5f42-48c8-85a0-848d53fa5056-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954431    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954731 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954438    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ede9ac93-5f42-48c8-85a0-848d53fa5056-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954731 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954447    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9p4t\" (UniqueName: \"kubernetes.io/projected/c73f4e32-0faa-48a1-8953-eca91366445c-kube-api-access-v9p4t\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954731 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954455    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73f4e32-0faa-48a1-8953-eca91366445c-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954731 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954463    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954731 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954475    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp7jd\" (UniqueName: \"kubernetes.io/projected/ede9ac93-5f42-48c8-85a0-848d53fa5056-kube-api-access-kp7jd\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:50.954731 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:50.954487    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/59552c85-6ea7-42f0-9b94-eb639f8f0dfd-kube-api-access-zwkrc\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:07:51.307715 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:51.307631    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"]
Nov 03 23:07:51.316504 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:51.316476    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd"]
Nov 03 23:07:51.429666 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:51.429630    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"]
Nov 03 23:07:51.434013 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:51.433992    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/premium-w0-68fbd1-kserve-85ccbcc869-grk9j"]
Nov 03 23:07:51.548217 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:51.548101    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"]
Nov 03 23:07:51.551976 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:51.551942    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-68fbd1/distinct-2-w0-68fbd1-kserve-64d65869bc-nh7gk"]
Nov 03 23:07:51.627091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:51.626984    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-68fbd1/simulator-w0-68fbd1-kserve-67fd977c44-blckd" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:07:52.854590 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:52.854562    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" path="/var/lib/kubelet/pods/59552c85-6ea7-42f0-9b94-eb639f8f0dfd/volumes"
Nov 03 23:07:52.855008 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:52.854993    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" path="/var/lib/kubelet/pods/c73f4e32-0faa-48a1-8953-eca91366445c/volumes"
Nov 03 23:07:52.855803 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:52.855782    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" path="/var/lib/kubelet/pods/ede9ac93-5f42-48c8-85a0-848d53fa5056/volumes"
Nov 03 23:07:53.115154 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:53.115048    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:07:57.921278 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:07:57.921237    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:02.719406 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:02.719361    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:04.698095 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:04.698063    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:04.896566 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:04.896522    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:05.429626 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:05.429583    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:06.937116 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:06.937080    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:08.455818 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:08.455783    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:08.734482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:08.734341    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:08.940009 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:08.939973    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:09.366288 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:09.366252    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:10.084346 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:08:10.084304    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:08:12.031154 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:12.031117    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:12.233998 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:12.233970    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:13.845073 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:13.845039    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:15.393611 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:15.393574    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:15.667556 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:15.667465    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:15.689469 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:15.689439    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"]
Nov 03 23:08:15.689642 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:15.689624    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy" containerID="cri-o://23702f64da24864b7cccff92b1ac4010383e9a55e92a13b530dbedbc8a01e907" gracePeriod=30
Nov 03 23:08:15.892253 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:15.892223    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:18.396500 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:18.396465    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:18.636357 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:18.636325    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:20.912680 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:20.912646    5162 generic.go:356] "Generic (PLEG): container finished" podID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerID="23702f64da24864b7cccff92b1ac4010383e9a55e92a13b530dbedbc8a01e907" exitCode=0
Nov 03 23:08:20.913064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:20.912713    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" event={"ID":"395a00c0-b9af-420d-9b3f-cf928586bff2","Type":"ContainerDied","Data":"23702f64da24864b7cccff92b1ac4010383e9a55e92a13b530dbedbc8a01e907"}
Nov 03 23:08:20.913064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:20.912772    5162 scope.go:122] "RemoveContainer" containerID="952c60156a37c0def551a62b0025fad0cfe7899ebfb362e5ed9bfd96dc32d876"
Nov 03 23:08:20.952559 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:20.952542    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 23:08:21.090859 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.090802    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/395a00c0-b9af-420d-9b3f-cf928586bff2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a00c0-b9af-420d-9b3f-cf928586bff2-istiod-ca-cert\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.090859 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.090840    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-credential-socket\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091033 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.090876    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-kube-api-access-f2f2k\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-kube-api-access-f2f2k\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091033 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.090903    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-token\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-token\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091033 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.090953    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-podinfo\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091033 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.090996    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-socket\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091033 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091023    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-certs\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091042    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-data\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091114    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-envoy\") pod \"395a00c0-b9af-420d-9b3f-cf928586bff2\" (UID: \"395a00c0-b9af-420d-9b3f-cf928586bff2\") "
Nov 03 23:08:21.091287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091175    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395a00c0-b9af-420d-9b3f-cf928586bff2-istiod-ca-cert" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:21.091287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091191    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-credential-socket" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.091531 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091293    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-socket" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.091592 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091525    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-certs" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.091592 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091548    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-data" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.091693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091634    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.091693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091652    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.091693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091664    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.091693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091672    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a00c0-b9af-420d-9b3f-cf928586bff2-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.091693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.091680    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.093210 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.093185    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-envoy" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.093304 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.093225    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-kube-api-access-f2f2k" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "kube-api-access-f2f2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:21.093471 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.093445    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-token" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:21.093569 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.093450    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-podinfo" pod "395a00c0-b9af-420d-9b3f-cf928586bff2" (UID: "395a00c0-b9af-420d-9b3f-cf928586bff2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:21.192483 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.192455    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.192483 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.192483    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2f2k\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-kube-api-access-f2f2k\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.192667 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.192498    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.192667 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.192509    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a00c0-b9af-420d-9b3f-cf928586bff2-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.920214 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.920184    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"
Nov 03 23:08:21.920214 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.920197    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d" event={"ID":"395a00c0-b9af-420d-9b3f-cf928586bff2","Type":"ContainerDied","Data":"3f8f4d0666c706e7284a36e43797688c674a097bb7ef979eb191d0db47cfe399"}
Nov 03 23:08:21.920705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:21.920251    5162 scope.go:122] "RemoveContainer" containerID="23702f64da24864b7cccff92b1ac4010383e9a55e92a13b530dbedbc8a01e907"
Nov 03 23:08:22.093572 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:22.093538    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"]
Nov 03 23:08:22.145016 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:22.144988    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d"]
Nov 03 23:08:22.734978 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:22.734944    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:22.853425 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:22.853376    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" path="/var/lib/kubelet/pods/395a00c0-b9af-420d-9b3f-cf928586bff2/volumes"
Nov 03 23:08:23.416498 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:23.416464    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:28.264690 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:28.264630    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:29.121663 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:29.121628    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"]
Nov 03 23:08:29.121916 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:29.121868    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy" containerID="cri-o://43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142" gracePeriod=30
Nov 03 23:08:32.385593 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:32.385556    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:32.513318 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:32.513282    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"]
Nov 03 23:08:32.513547 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:32.513524    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" containerID="cri-o://cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15" gracePeriod=30
Nov 03 23:08:33.121813 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:33.121764    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:33.159212 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:33.159177    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:08:33.159360 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:33.159245    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:08:34.376003 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.375981    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 23:08:34.433162 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433135    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-token\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-token\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433295 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433172    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-data\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433295 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433192    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-credential-socket\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433295 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433229    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-envoy\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433348    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-socket\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433431    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-certs\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433483    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-kube-api-access-d5bpt\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-kube-api-access-d5bpt\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433650 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433521    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/857d62dc-6b25-4d48-b706-d995c5fc2fef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857d62dc-6b25-4d48-b706-d995c5fc2fef-istiod-ca-cert\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433650 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433540    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-data" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:34.433650 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433563    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-credential-socket" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:34.433650 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433592    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-podinfo\") pod \"857d62dc-6b25-4d48-b706-d995c5fc2fef\" (UID: \"857d62dc-6b25-4d48-b706-d995c5fc2fef\") "
Nov 03 23:08:34.433868 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433760    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-certs" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:34.433868 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.433771    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-socket" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:34.434069 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.434035    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/857d62dc-6b25-4d48-b706-d995c5fc2fef-istiod-ca-cert" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:34.434171 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.434133    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.434171 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.434156    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.434171 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.434170    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.434293 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.434183    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.435592 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.435573    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-kube-api-access-d5bpt" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "kube-api-access-d5bpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:34.435761 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.435739    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-token" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:34.435977 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.435957    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-podinfo" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:34.436076 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.436059    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-envoy" pod "857d62dc-6b25-4d48-b706-d995c5fc2fef" (UID: "857d62dc-6b25-4d48-b706-d995c5fc2fef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:34.534866 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.534841    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5bpt\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-kube-api-access-d5bpt\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.534866 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.534864    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857d62dc-6b25-4d48-b706-d995c5fc2fef-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.535020 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.534873    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.535020 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.534883    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.535020 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.534891    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857d62dc-6b25-4d48-b706-d995c5fc2fef-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:34.992802 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.992772    5162 generic.go:356] "Generic (PLEG): container finished" podID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerID="43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142" exitCode=0
Nov 03 23:08:34.992954 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.992816    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"
Nov 03 23:08:34.992954 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.992851    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" event={"ID":"857d62dc-6b25-4d48-b706-d995c5fc2fef","Type":"ContainerDied","Data":"43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142"}
Nov 03 23:08:34.992954 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.992907    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm" event={"ID":"857d62dc-6b25-4d48-b706-d995c5fc2fef","Type":"ContainerDied","Data":"3bf3a688a707d5841c3a672597b1e998f39eae873bec73f4b2de02acb275f82a"}
Nov 03 23:08:34.992954 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:34.992935    5162 scope.go:122] "RemoveContainer" containerID="43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142"
Nov 03 23:08:35.001927 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:35.001908    5162 scope.go:122] "RemoveContainer" containerID="43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142"
Nov 03 23:08:35.002194 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:08:35.002172    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142\": container with ID starting with 43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142 not found: ID does not exist" containerID="43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142"
Nov 03 23:08:35.002269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:35.002201    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142"} err="failed to get container status \"43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142\": rpc error: code = NotFound desc = could not find container \"43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142\": container with ID starting with 43fdc8a4c9b2826601ff02718e2354c757516a74b6d016119c5b14ae17883142 not found: ID does not exist"
Nov 03 23:08:35.164158 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:35.164076    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"]
Nov 03 23:08:35.215314 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:35.215284    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm"]
Nov 03 23:08:36.853602 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:36.853563    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" path="/var/lib/kubelet/pods/857d62dc-6b25-4d48-b706-d995c5fc2fef/volumes"
Nov 03 23:08:37.750835 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.750812    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 23:08:37.866758 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.866693    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2911c96-53cb-4128-8f79-0975e1b97894-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2911c96-53cb-4128-8f79-0975e1b97894-istiod-ca-cert\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.866755    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-socket\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.866804    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-certs\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.866865    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-credential-socket\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.866885    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-envoy\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.866928    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-istio-token\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-istio-token\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.866984    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-data\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867020    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-kube-api-access-4hgj4\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-kube-api-access-4hgj4\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867048    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a2911c96-53cb-4128-8f79-0975e1b97894-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2911c96-53cb-4128-8f79-0975e1b97894-istio-podinfo\") pod \"a2911c96-53cb-4128-8f79-0975e1b97894\" (UID: \"a2911c96-53cb-4128-8f79-0975e1b97894\") "
Nov 03 23:08:37.867175 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867047    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-socket" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867150    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-credential-socket" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867201    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-certs" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867247    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2911c96-53cb-4128-8f79-0975e1b97894-istiod-ca-cert" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867365    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-data" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867592    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867612    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2911c96-53cb-4128-8f79-0975e1b97894-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867628    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867642    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.867656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.867657    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.869011 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.868983    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-envoy" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:37.869137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.869119    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2911c96-53cb-4128-8f79-0975e1b97894-istio-podinfo" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:37.869278 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.869260    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-kube-api-access-4hgj4" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "kube-api-access-4hgj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:37.869334 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.869276    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-istio-token" pod "a2911c96-53cb-4128-8f79-0975e1b97894" (UID: "a2911c96-53cb-4128-8f79-0975e1b97894"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:37.917901 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.917804    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:37.968612 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.968581    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2911c96-53cb-4128-8f79-0975e1b97894-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.968612 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.968606    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.968612 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.968617    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hgj4\" (UniqueName: \"kubernetes.io/projected/a2911c96-53cb-4128-8f79-0975e1b97894-kube-api-access-4hgj4\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.968839 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.968627    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2911c96-53cb-4128-8f79-0975e1b97894-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:37.989293 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.989267    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"]
Nov 03 23:08:37.989588 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:37.989474    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs" podUID="c3268422-a2de-48be-ba19-1860bead948c" containerName="discovery" containerID="cri-o://af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143" gracePeriod=30
Nov 03 23:08:38.016107 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.016076    5162 generic.go:356] "Generic (PLEG): container finished" podID="a2911c96-53cb-4128-8f79-0975e1b97894" containerID="cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15" exitCode=0
Nov 03 23:08:38.016205 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.016122    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"
Nov 03 23:08:38.016205 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.016153    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" event={"ID":"a2911c96-53cb-4128-8f79-0975e1b97894","Type":"ContainerDied","Data":"cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15"}
Nov 03 23:08:38.016271 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.016205    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj" event={"ID":"a2911c96-53cb-4128-8f79-0975e1b97894","Type":"ContainerDied","Data":"9c2d86935374fe01a6356fb5a01ccd59c0a4d0a6823af567d216b647d50aa793"}
Nov 03 23:08:38.016271 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.016234    5162 scope.go:122] "RemoveContainer" containerID="cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15"
Nov 03 23:08:38.102293 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.102269    5162 scope.go:122] "RemoveContainer" containerID="cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15"
Nov 03 23:08:38.102660 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:08:38.102636    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15\": container with ID starting with cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15 not found: ID does not exist" containerID="cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15"
Nov 03 23:08:38.102735 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.102671    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15"} err="failed to get container status \"cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15\": rpc error: code = NotFound desc = could not find container \"cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15\": container with ID starting with cd354f63175224e07efe1ad76271e0bacf92442710500e06b40841275faffd15 not found: ID does not exist"
Nov 03 23:08:38.231710 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.231682    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"]
Nov 03 23:08:38.234403 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.234373    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 23:08:38.235632 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.235615    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj"]
Nov 03 23:08:38.272142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272114    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-dns-cert\") pod \"c3268422-a2de-48be-ba19-1860bead948c\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") "
Nov 03 23:08:38.272299 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272163    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-kube-api-access-wpn25\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-kube-api-access-wpn25\") pod \"c3268422-a2de-48be-ba19-1860bead948c\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") "
Nov 03 23:08:38.272299 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272214    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-cacerts\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-cacerts\") pod \"c3268422-a2de-48be-ba19-1860bead948c\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") "
Nov 03 23:08:38.272299 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272238    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3268422-a2de-48be-ba19-1860bead948c-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3268422-a2de-48be-ba19-1860bead948c-local-certs\") pod \"c3268422-a2de-48be-ba19-1860bead948c\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") "
Nov 03 23:08:38.272495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272315    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-ca-configmap\") pod \"c3268422-a2de-48be-ba19-1860bead948c\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") "
Nov 03 23:08:38.272495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272367    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-istio-token\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-istio-token\") pod \"c3268422-a2de-48be-ba19-1860bead948c\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") "
Nov 03 23:08:38.272603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272492    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-kubeconfig\") pod \"c3268422-a2de-48be-ba19-1860bead948c\" (UID: \"c3268422-a2de-48be-ba19-1860bead948c\") "
Nov 03 23:08:38.272976 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272728    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-ca-configmap" pod "c3268422-a2de-48be-ba19-1860bead948c" (UID: "c3268422-a2de-48be-ba19-1860bead948c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:38.272976 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.272941    5162 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-ca-configmap\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.275065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.275040    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3268422-a2de-48be-ba19-1860bead948c-local-certs" pod "c3268422-a2de-48be-ba19-1860bead948c" (UID: "c3268422-a2de-48be-ba19-1860bead948c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:38.275187 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.275165    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-kube-api-access-wpn25" pod "c3268422-a2de-48be-ba19-1860bead948c" (UID: "c3268422-a2de-48be-ba19-1860bead948c"). InnerVolumeSpecName "kube-api-access-wpn25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:38.275310 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.275295    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-dns-cert" pod "c3268422-a2de-48be-ba19-1860bead948c" (UID: "c3268422-a2de-48be-ba19-1860bead948c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:38.275442 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.275425    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-kubeconfig" pod "c3268422-a2de-48be-ba19-1860bead948c" (UID: "c3268422-a2de-48be-ba19-1860bead948c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:38.275520 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.275499    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-istio-token" pod "c3268422-a2de-48be-ba19-1860bead948c" (UID: "c3268422-a2de-48be-ba19-1860bead948c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:38.275618 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.275603    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-cacerts" pod "c3268422-a2de-48be-ba19-1860bead948c" (UID: "c3268422-a2de-48be-ba19-1860bead948c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:38.374426 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.374376    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.374426 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.374418    5162 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-kubeconfig\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.374426 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.374431    5162 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-istio-csr-dns-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.374656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.374443    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpn25\" (UniqueName: \"kubernetes.io/projected/c3268422-a2de-48be-ba19-1860bead948c-kube-api-access-wpn25\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.374656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.374458    5162 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3268422-a2de-48be-ba19-1860bead948c-cacerts\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.374656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.374469    5162 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3268422-a2de-48be-ba19-1860bead948c-local-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.853851 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:38.853781    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" path="/var/lib/kubelet/pods/a2911c96-53cb-4128-8f79-0975e1b97894/volumes"
Nov 03 23:08:39.023924 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.023892    5162 generic.go:356] "Generic (PLEG): container finished" podID="c3268422-a2de-48be-ba19-1860bead948c" containerID="af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143" exitCode=0
Nov 03 23:08:39.024339 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.023934    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"
Nov 03 23:08:39.024339 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.023976    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs" event={"ID":"c3268422-a2de-48be-ba19-1860bead948c","Type":"ContainerDied","Data":"af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143"}
Nov 03 23:08:39.024339 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.024023    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs" event={"ID":"c3268422-a2de-48be-ba19-1860bead948c","Type":"ContainerDied","Data":"66a2af47e81463b09d77e18ff6281e32eb62a23274ba6088c4da39c2ea3c7fdd"}
Nov 03 23:08:39.024339 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.024043    5162 scope.go:122] "RemoveContainer" containerID="af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143"
Nov 03 23:08:39.033700 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.033682    5162 scope.go:122] "RemoveContainer" containerID="af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143"
Nov 03 23:08:39.033977 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:08:39.033959    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143\": container with ID starting with af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143 not found: ID does not exist" containerID="af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143"
Nov 03 23:08:39.034041 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.033985    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143"} err="failed to get container status \"af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143\": rpc error: code = NotFound desc = could not find container \"af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143\": container with ID starting with af123b9765365ea5440c9b1b2d7fbb9d6d7ed147ba71a515e45778ecb3062143 not found: ID does not exist"
Nov 03 23:08:39.211801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.211775    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"]
Nov 03 23:08:39.264630 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:39.264604    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs"]
Nov 03 23:08:40.083068 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:08:40.083031    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:08:40.854037 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:40.854000    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3268422-a2de-48be-ba19-1860bead948c" path="/var/lib/kubelet/pods/c3268422-a2de-48be-ba19-1860bead948c/volumes"
Nov 03 23:08:42.761372 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:42.761339    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:47.645142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:47.645110    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:49.812727 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:49.812685    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:50.018579 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:50.018540    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:52.398979 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:52.398944    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:08:57.214894 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:08:57.214858    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:02.125547 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:02.125501    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:06.922985 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:06.922940    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:10.082734 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:09:10.082689    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:09:10.597546 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:10.597512    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:11.741513 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:11.741477    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:16.518994 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:16.518963    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:18.752456 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:18.752429    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:21.329948 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:21.329912    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:23.086229 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:23.086189    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:26.106739 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:26.106689    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:30.917805 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:30.917764    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:35.712419 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:35.712359    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:38.514884 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:38.514836    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:40.084407 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:09:40.084359    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:09:40.596604 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:40.596571    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:42.811517 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:42.811485    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:45.815761 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:45.815724    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:50.094793 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:50.094756    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:50.606166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:50.606120    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:50.854785 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:50.854746    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:51.924405 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:51.924357    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:54.671281 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:54.671247    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:54.955141 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:54.955058    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:09:57.149819 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:09:57.149786    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:02.130643 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:02.130606    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:05.733963 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:05.733921    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:07.004947 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:07.004909    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:10.083080 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:10:10.083041    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:10:11.711216 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:11.711179    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:11.916659 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:11.916625    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:12.119129 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:12.119042    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:12.806433 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:12.806376    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:16.941859 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:16.941825    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:19.105944 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:19.105911    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:21.912814 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:21.912786    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:23.339853 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:23.339808    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:23.589156 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:23.589117    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:24.340762 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:24.340727    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:24.798741 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:24.798698    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:24.999984 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:24.999948    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:26.712685 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:26.712651    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:31.520988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:31.520955    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:32.306367 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:32.306333    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:32.552960 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:32.552915    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:35.576847 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.576812    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"]
Nov 03 23:10:35.577260 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.577027    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x" podUID="abd9e7fd-4f46-48ba-9947-75a14210c989" containerName="maas-api" containerID="cri-o://3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08" gracePeriod=30
Nov 03 23:10:35.727884 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.727850    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:35.828495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.828420    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:10:35.986912 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.986880    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-metrics-certs\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-metrics-certs\") pod \"abd9e7fd-4f46-48ba-9947-75a14210c989\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") "
Nov 03 23:10:35.987082 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.986929    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abd9e7fd-4f46-48ba-9947-75a14210c989-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd9e7fd-4f46-48ba-9947-75a14210c989-openshift-service-ca\") pod \"abd9e7fd-4f46-48ba-9947-75a14210c989\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") "
Nov 03 23:10:35.987082 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.987023    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls\") pod \"abd9e7fd-4f46-48ba-9947-75a14210c989\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") "
Nov 03 23:10:35.987156 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.987087    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abd9e7fd-4f46-48ba-9947-75a14210c989-kube-api-access-zzppr\" (UniqueName: \"kubernetes.io/projected/abd9e7fd-4f46-48ba-9947-75a14210c989-kube-api-access-zzppr\") pod \"abd9e7fd-4f46-48ba-9947-75a14210c989\" (UID: \"abd9e7fd-4f46-48ba-9947-75a14210c989\") "
Nov 03 23:10:35.987325 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.987298    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abd9e7fd-4f46-48ba-9947-75a14210c989-openshift-service-ca" pod "abd9e7fd-4f46-48ba-9947-75a14210c989" (UID: "abd9e7fd-4f46-48ba-9947-75a14210c989"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:10:35.989166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.989141    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls" pod "abd9e7fd-4f46-48ba-9947-75a14210c989" (UID: "abd9e7fd-4f46-48ba-9947-75a14210c989"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:10:35.989166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.989165    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-metrics-certs" pod "abd9e7fd-4f46-48ba-9947-75a14210c989" (UID: "abd9e7fd-4f46-48ba-9947-75a14210c989"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:10:35.989299 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:35.989233    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abd9e7fd-4f46-48ba-9947-75a14210c989-kube-api-access-zzppr" pod "abd9e7fd-4f46-48ba-9947-75a14210c989" (UID: "abd9e7fd-4f46-48ba-9947-75a14210c989"). InnerVolumeSpecName "kube-api-access-zzppr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:10:36.088725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.088650    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzppr\" (UniqueName: \"kubernetes.io/projected/abd9e7fd-4f46-48ba-9947-75a14210c989-kube-api-access-zzppr\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:36.088725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.088677    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:36.088725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.088687    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd9e7fd-4f46-48ba-9947-75a14210c989-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:36.088725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.088697    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd9e7fd-4f46-48ba-9947-75a14210c989-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:36.661826 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.661796    5162 generic.go:356] "Generic (PLEG): container finished" podID="abd9e7fd-4f46-48ba-9947-75a14210c989" containerID="3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08" exitCode=0
Nov 03 23:10:36.662242 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.661836    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"
Nov 03 23:10:36.662242 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.661841    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x" event={"ID":"abd9e7fd-4f46-48ba-9947-75a14210c989","Type":"ContainerDied","Data":"3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08"}
Nov 03 23:10:36.662242 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.661885    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x" event={"ID":"abd9e7fd-4f46-48ba-9947-75a14210c989","Type":"ContainerDied","Data":"5b4ed8502438f5f7f1221e59bfb72bbfd7c4b4396a83d7f70d1bf96194d4c73e"}
Nov 03 23:10:36.662242 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.661905    5162 scope.go:122] "RemoveContainer" containerID="3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08"
Nov 03 23:10:36.671509 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.671382    5162 scope.go:122] "RemoveContainer" containerID="3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08"
Nov 03 23:10:36.671765 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:10:36.671747    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08\": container with ID starting with 3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08 not found: ID does not exist" containerID="3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08"
Nov 03 23:10:36.671829 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.671773    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08"} err="failed to get container status \"3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08\": rpc error: code = NotFound desc = could not find container \"3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08\": container with ID starting with 3317003578c42d50dc55f926011330f1c2e97f0233d03278eb29ce7289421d08 not found: ID does not exist"
Nov 03 23:10:36.831196 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.831156    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"]
Nov 03 23:10:36.834941 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.834917    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ac6633-55b54c97b5-9hx8x"]
Nov 03 23:10:36.853168 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:36.853141    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abd9e7fd-4f46-48ba-9947-75a14210c989" path="/var/lib/kubelet/pods/abd9e7fd-4f46-48ba-9947-75a14210c989/volumes"
Nov 03 23:10:37.241431 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:37.241384    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:37.515331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:37.515248    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:37.715882 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:37.715847    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:38.171261 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:38.171228    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:40.084262 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:10:40.084228    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:10:42.613349 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:42.613315    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:43.166325 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:43.166284    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:44.063153 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:44.063114    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:44.313730 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:44.313633    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:44.334796 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:44.334767    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"]
Nov 03 23:10:44.335006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:44.334984    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy" containerID="cri-o://6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806" gracePeriod=30
Nov 03 23:10:44.514542 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:44.514498    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:44.856112 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:44.856074    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:46.205756 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:46.205720    5162 patch_prober.go:29] interesting pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:10:46.206133 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:46.205771    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:10:47.700642 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:47.700598    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:49.590671 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.590647    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:10:49.629963 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.629895    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-token\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-token\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.629963 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.629953    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-podinfo\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630000    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-certs\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630039    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-credential-socket\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630070    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-socket\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630099    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-data\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630179    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-kube-api-access-7pg78\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-kube-api-access-7pg78\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630224    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-envoy\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630296    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istiod-ca-cert\") pod \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\" (UID: \"013e4f6a-2cf6-4d66-8580-77c5e2359f53\") "
Nov 03 23:10:49.630370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630296    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-certs" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:10:49.630370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630307    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-credential-socket" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:10:49.630370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630321    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-socket" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:10:49.630664 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630469    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-data" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:10:49.630737 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630715    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istiod-ca-cert" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:10:49.630804 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630789    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.630862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630813    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.630862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630828    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.630862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.630843    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.632137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.632112    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-token" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:10:49.632252 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.632208    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-podinfo" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:10:49.632252 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.632216    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-kube-api-access-7pg78" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "kube-api-access-7pg78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:10:49.632335 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.632300    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-envoy" pod "013e4f6a-2cf6-4d66-8580-77c5e2359f53" (UID: "013e4f6a-2cf6-4d66-8580-77c5e2359f53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:10:49.731996 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.731969    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.731996 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.731995    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pg78\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-kube-api-access-7pg78\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.731996 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.732005    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.732236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.732013    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.732236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.732021    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013e4f6a-2cf6-4d66-8580-77c5e2359f53-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:10:49.734476 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.734452    5162 generic.go:356] "Generic (PLEG): container finished" podID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerID="6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806" exitCode=0
Nov 03 23:10:49.734603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.734501    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"
Nov 03 23:10:49.734603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.734543    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" event={"ID":"013e4f6a-2cf6-4d66-8580-77c5e2359f53","Type":"ContainerDied","Data":"6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806"}
Nov 03 23:10:49.734724 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.734612    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v" event={"ID":"013e4f6a-2cf6-4d66-8580-77c5e2359f53","Type":"ContainerDied","Data":"98c8fdd49633422110a9dae4daf680e1ff71a76f7289221073532dc7012d0dc0"}
Nov 03 23:10:49.734724 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.734648    5162 scope.go:122] "RemoveContainer" containerID="6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806"
Nov 03 23:10:49.744380 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.744365    5162 scope.go:122] "RemoveContainer" containerID="2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55"
Nov 03 23:10:49.753368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.753351    5162 scope.go:122] "RemoveContainer" containerID="6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806"
Nov 03 23:10:49.753636 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:10:49.753617    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806\": container with ID starting with 6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806 not found: ID does not exist" containerID="6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806"
Nov 03 23:10:49.753723 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.753648    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806"} err="failed to get container status \"6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806\": rpc error: code = NotFound desc = could not find container \"6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806\": container with ID starting with 6913f0c48e49d314f98d3fbd2f9d438902f54bffec0e094b004d60e193c42806 not found: ID does not exist"
Nov 03 23:10:49.753723 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.753678    5162 scope.go:122] "RemoveContainer" containerID="2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55"
Nov 03 23:10:49.753965 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:10:49.753946    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55\": container with ID starting with 2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55 not found: ID does not exist" containerID="2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55"
Nov 03 23:10:49.754025 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.753971    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55"} err="failed to get container status \"2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55\": rpc error: code = NotFound desc = could not find container \"2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55\": container with ID starting with 2de78f3f0ce77362bba46d279f2012b5c1c14d3d4b55d395ea6ffcec84a32e55 not found: ID does not exist"
Nov 03 23:10:49.761285 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.761265    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"]
Nov 03 23:10:49.815003 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.814976    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"]
Nov 03 23:10:49.820303 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.820277    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"]
Nov 03 23:10:49.882580 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.882505    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"]
Nov 03 23:10:49.886570 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.886546    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v"]
Nov 03 23:10:49.977235 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:49.977199    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:50.853936 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:50.853905    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" path="/var/lib/kubelet/pods/013e4f6a-2cf6-4d66-8580-77c5e2359f53/volumes"
Nov 03 23:10:52.708362 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:52.708324    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:57.299526 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:57.298912    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:10:57.553033 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:10:57.552945    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:04.780094 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:04.780044    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main" containerID="cri-o://478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde" gracePeriod=45
Nov 03 23:11:04.835413 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:04.835343    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" containerID="cri-o://95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c" gracePeriod=45
Nov 03 23:11:04.838335 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:04.838307    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" containerID="cri-o://d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c" gracePeriod=45
Nov 03 23:11:04.942733 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:04.942694    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": dial tcp 10.134.0.52:8000: connect: connection refused"
Nov 03 23:11:04.973868 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:04.973817    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": dial tcp 10.134.0.55:8000: connect: connection refused"
Nov 03 23:11:05.088096 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.088068    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 23:11:05.107652 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.107623    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-home\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-home\") pod \"fe0cad46-96b3-44fb-81a4-9415d4db7046\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") "
Nov 03 23:11:05.107828 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.107683    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe0cad46-96b3-44fb-81a4-9415d4db7046-kube-api-access-5f2fp\" (UniqueName: \"kubernetes.io/projected/fe0cad46-96b3-44fb-81a4-9415d4db7046-kube-api-access-5f2fp\") pod \"fe0cad46-96b3-44fb-81a4-9415d4db7046\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") "
Nov 03 23:11:05.107828 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.107727    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-tmp-dir\") pod \"fe0cad46-96b3-44fb-81a4-9415d4db7046\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") "
Nov 03 23:11:05.107828 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.107745    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe0cad46-96b3-44fb-81a4-9415d4db7046-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0cad46-96b3-44fb-81a4-9415d4db7046-tls-certs\") pod \"fe0cad46-96b3-44fb-81a4-9415d4db7046\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") "
Nov 03 23:11:05.107828 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.107806    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-model-cache\") pod \"fe0cad46-96b3-44fb-81a4-9415d4db7046\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") "
Nov 03 23:11:05.108064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.107854    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-dshm\") pod \"fe0cad46-96b3-44fb-81a4-9415d4db7046\" (UID: \"fe0cad46-96b3-44fb-81a4-9415d4db7046\") "
Nov 03 23:11:05.108064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.107918    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-home" pod "fe0cad46-96b3-44fb-81a4-9415d4db7046" (UID: "fe0cad46-96b3-44fb-81a4-9415d4db7046"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.108224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.108204    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.108532 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.108507    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-model-cache" pod "fe0cad46-96b3-44fb-81a4-9415d4db7046" (UID: "fe0cad46-96b3-44fb-81a4-9415d4db7046"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.108670 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.108618    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-tmp-dir" pod "fe0cad46-96b3-44fb-81a4-9415d4db7046" (UID: "fe0cad46-96b3-44fb-81a4-9415d4db7046"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.110292 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.110266    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-dshm" pod "fe0cad46-96b3-44fb-81a4-9415d4db7046" (UID: "fe0cad46-96b3-44fb-81a4-9415d4db7046"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.110386 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.110358    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe0cad46-96b3-44fb-81a4-9415d4db7046-tls-certs" pod "fe0cad46-96b3-44fb-81a4-9415d4db7046" (UID: "fe0cad46-96b3-44fb-81a4-9415d4db7046"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:11:05.110498 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.110482    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe0cad46-96b3-44fb-81a4-9415d4db7046-kube-api-access-5f2fp" pod "fe0cad46-96b3-44fb-81a4-9415d4db7046" (UID: "fe0cad46-96b3-44fb-81a4-9415d4db7046"). InnerVolumeSpecName "kube-api-access-5f2fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:11:05.122863 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.122839    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 23:11:05.124694 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.124679    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 23:11:05.208719 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.208685    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a193c2a-098b-49bd-9924-0b6b3fc70718-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a193c2a-098b-49bd-9924-0b6b3fc70718-tls-certs\") pod \"9a193c2a-098b-49bd-9924-0b6b3fc70718\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") "
Nov 03 23:11:05.208903 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.208739    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-home\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-home\") pod \"9a193c2a-098b-49bd-9924-0b6b3fc70718\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") "
Nov 03 23:11:05.208903 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.208780    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tmp-dir\") pod \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") "
Nov 03 23:11:05.208903 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.208833    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tls-certs\" (UniqueName: \"kubernetes.io/secret/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tls-certs\") pod \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") "
Nov 03 23:11:05.208903 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.208873    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-dshm\") pod \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") "
Nov 03 23:11:05.209132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.208908    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-tmp-dir\") pod \"9a193c2a-098b-49bd-9924-0b6b3fc70718\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") "
Nov 03 23:11:05.209132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.208972    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a193c2a-098b-49bd-9924-0b6b3fc70718-kube-api-access-6znsz\" (UniqueName: \"kubernetes.io/projected/9a193c2a-098b-49bd-9924-0b6b3fc70718-kube-api-access-6znsz\") pod \"9a193c2a-098b-49bd-9924-0b6b3fc70718\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") "
Nov 03 23:11:05.209132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209022    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-model-cache\") pod \"9a193c2a-098b-49bd-9924-0b6b3fc70718\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") "
Nov 03 23:11:05.209132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209054    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-home\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-home\") pod \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") "
Nov 03 23:11:05.209132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209099    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-dshm\") pod \"9a193c2a-098b-49bd-9924-0b6b3fc70718\" (UID: \"9a193c2a-098b-49bd-9924-0b6b3fc70718\") "
Nov 03 23:11:05.209132 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209131    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-kube-api-access-f4qxc\" (UniqueName: \"kubernetes.io/projected/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-kube-api-access-f4qxc\") pod \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") "
Nov 03 23:11:05.209370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209207    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-model-cache\") pod \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\" (UID: \"f923fc18-ec7e-4028-865c-c6e0f1b40f3f\") "
Nov 03 23:11:05.209370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209209    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tmp-dir" pod "f923fc18-ec7e-4028-865c-c6e0f1b40f3f" (UID: "f923fc18-ec7e-4028-865c-c6e0f1b40f3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.209370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209288    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-home" pod "9a193c2a-098b-49bd-9924-0b6b3fc70718" (UID: "9a193c2a-098b-49bd-9924-0b6b3fc70718"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.209370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209304    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-tmp-dir" pod "9a193c2a-098b-49bd-9924-0b6b3fc70718" (UID: "9a193c2a-098b-49bd-9924-0b6b3fc70718"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.209539 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209522    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-model-cache" pod "f923fc18-ec7e-4028-865c-c6e0f1b40f3f" (UID: "f923fc18-ec7e-4028-865c-c6e0f1b40f3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.210125 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.209855    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-home" pod "f923fc18-ec7e-4028-865c-c6e0f1b40f3f" (UID: "f923fc18-ec7e-4028-865c-c6e0f1b40f3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210349    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210376    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0cad46-96b3-44fb-81a4-9415d4db7046-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210406    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210421    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210435    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210448    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210463    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0cad46-96b3-44fb-81a4-9415d4db7046-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210475    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210490    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f2fp\" (UniqueName: \"kubernetes.io/projected/fe0cad46-96b3-44fb-81a4-9415d4db7046-kube-api-access-5f2fp\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.210522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.210503    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.211153 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.211123    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-model-cache" pod "9a193c2a-098b-49bd-9924-0b6b3fc70718" (UID: "9a193c2a-098b-49bd-9924-0b6b3fc70718"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.212705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.211866    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a193c2a-098b-49bd-9924-0b6b3fc70718-tls-certs" pod "9a193c2a-098b-49bd-9924-0b6b3fc70718" (UID: "9a193c2a-098b-49bd-9924-0b6b3fc70718"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:11:05.212705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.212037    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a193c2a-098b-49bd-9924-0b6b3fc70718-kube-api-access-6znsz" pod "9a193c2a-098b-49bd-9924-0b6b3fc70718" (UID: "9a193c2a-098b-49bd-9924-0b6b3fc70718"). InnerVolumeSpecName "kube-api-access-6znsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:11:05.212705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.212447    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-dshm" pod "9a193c2a-098b-49bd-9924-0b6b3fc70718" (UID: "9a193c2a-098b-49bd-9924-0b6b3fc70718"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.212705 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.212638    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tls-certs" pod "f923fc18-ec7e-4028-865c-c6e0f1b40f3f" (UID: "f923fc18-ec7e-4028-865c-c6e0f1b40f3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:11:05.212984 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.212895    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-dshm" pod "f923fc18-ec7e-4028-865c-c6e0f1b40f3f" (UID: "f923fc18-ec7e-4028-865c-c6e0f1b40f3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:05.214029 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.214001    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-kube-api-access-f4qxc" pod "f923fc18-ec7e-4028-865c-c6e0f1b40f3f" (UID: "f923fc18-ec7e-4028-865c-c6e0f1b40f3f"). InnerVolumeSpecName "kube-api-access-f4qxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:11:05.311801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.311712    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.311801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.311751    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a193c2a-098b-49bd-9924-0b6b3fc70718-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.311801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.311771    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4qxc\" (UniqueName: \"kubernetes.io/projected/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-kube-api-access-f4qxc\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.311801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.311789    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a193c2a-098b-49bd-9924-0b6b3fc70718-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.311801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.311802    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.312065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.311816    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f923fc18-ec7e-4028-865c-c6e0f1b40f3f-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.312065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.311830    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6znsz\" (UniqueName: \"kubernetes.io/projected/9a193c2a-098b-49bd-9924-0b6b3fc70718-kube-api-access-6znsz\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:05.819191 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.819159    5162 generic.go:356] "Generic (PLEG): container finished" podID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerID="478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde" exitCode=0
Nov 03 23:11:05.819608 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.819210    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"
Nov 03 23:11:05.819608 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.819248    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" event={"ID":"fe0cad46-96b3-44fb-81a4-9415d4db7046","Type":"ContainerDied","Data":"478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde"}
Nov 03 23:11:05.819608 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.819291    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2" event={"ID":"fe0cad46-96b3-44fb-81a4-9415d4db7046","Type":"ContainerDied","Data":"eff0de5d6834feef7bec578c28d93ad8d366f47fab80142c2d41becbffc65841"}
Nov 03 23:11:05.819608 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.819312    5162 scope.go:122] "RemoveContainer" containerID="478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde"
Nov 03 23:11:05.820716 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.820688    5162 generic.go:356] "Generic (PLEG): container finished" podID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerID="95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c" exitCode=0
Nov 03 23:11:05.820801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.820734    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"
Nov 03 23:11:05.820801 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.820781    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" event={"ID":"9a193c2a-098b-49bd-9924-0b6b3fc70718","Type":"ContainerDied","Data":"95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c"}
Nov 03 23:11:05.820904 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.820822    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt" event={"ID":"9a193c2a-098b-49bd-9924-0b6b3fc70718","Type":"ContainerDied","Data":"77dd732ab0ff579189428403859ee6afb57a74e155935f76dc5b9d21c0976b1a"}
Nov 03 23:11:05.821946 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.821925    5162 generic.go:356] "Generic (PLEG): container finished" podID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerID="d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c" exitCode=0
Nov 03 23:11:05.822036 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.821990    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" event={"ID":"f923fc18-ec7e-4028-865c-c6e0f1b40f3f","Type":"ContainerDied","Data":"d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c"}
Nov 03 23:11:05.822036 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.822009    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr" event={"ID":"f923fc18-ec7e-4028-865c-c6e0f1b40f3f","Type":"ContainerDied","Data":"43e407a5fdd21fbcaf803e24a348b3c8dbdaf8ddf628694f3b6512978f3cae62"}
Nov 03 23:11:05.822124 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.822047    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"
Nov 03 23:11:05.829734 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.829714    5162 scope.go:122] "RemoveContainer" containerID="478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde"
Nov 03 23:11:05.829974 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:11:05.829956    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde\": container with ID starting with 478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde not found: ID does not exist" containerID="478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde"
Nov 03 23:11:05.830053 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.829982    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde"} err="failed to get container status \"478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde\": rpc error: code = NotFound desc = could not find container \"478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde\": container with ID starting with 478d44fca6e8d97668bb15310b58db0724c7339d39821115451b9dfedb387fde not found: ID does not exist"
Nov 03 23:11:05.830053 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.829997    5162 scope.go:122] "RemoveContainer" containerID="95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c"
Nov 03 23:11:05.839532 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.839509    5162 scope.go:122] "RemoveContainer" containerID="95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c"
Nov 03 23:11:05.839778 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:11:05.839761    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c\": container with ID starting with 95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c not found: ID does not exist" containerID="95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c"
Nov 03 23:11:05.839826 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.839785    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c"} err="failed to get container status \"95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c\": rpc error: code = NotFound desc = could not find container \"95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c\": container with ID starting with 95df2e544091291b39dc3e854d2b5431ff8ed40aeaf21cbb52f79d2deb439d6c not found: ID does not exist"
Nov 03 23:11:05.839826 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.839800    5162 scope.go:122] "RemoveContainer" containerID="d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c"
Nov 03 23:11:05.847378 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.847361    5162 scope.go:122] "RemoveContainer" containerID="d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c"
Nov 03 23:11:05.847634 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:11:05.847615    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c\": container with ID starting with d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c not found: ID does not exist" containerID="d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c"
Nov 03 23:11:05.847685 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.847640    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c"} err="failed to get container status \"d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c\": rpc error: code = NotFound desc = could not find container \"d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c\": container with ID starting with d84d60bf41b2d6137dd6b2bd0b3722c355a47f803f829106380edd24f4d6c23c not found: ID does not exist"
Nov 03 23:11:05.988878 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:05.988848    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"]
Nov 03 23:11:06.040177 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.040148    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/distinct-2-w5-6f1190-kserve-6c449fdd4d-tvzb2"]
Nov 03 23:11:06.105343 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.105269    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"]
Nov 03 23:11:06.110858 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.110832    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/simulator-w5-6f1190-kserve-6885f5f869-lm2dr"]
Nov 03 23:11:06.222852 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.222814    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"]
Nov 03 23:11:06.226933 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.226908    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-6f1190/premium-w5-6f1190-kserve-7974cbc8f6-b8lxt"]
Nov 03 23:11:06.852338 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.852305    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" path="/var/lib/kubelet/pods/9a193c2a-098b-49bd-9924-0b6b3fc70718/volumes"
Nov 03 23:11:06.852771 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.852757    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" path="/var/lib/kubelet/pods/f923fc18-ec7e-4028-865c-c6e0f1b40f3f/volumes"
Nov 03 23:11:06.853147 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:06.853135    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" path="/var/lib/kubelet/pods/fe0cad46-96b3-44fb-81a4-9415d4db7046/volumes"
Nov 03 23:11:10.082873 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:11:10.082836    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:11:23.816187 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:23.816146    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:26.767632 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:26.767599    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:26.987478 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:26.987432    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:27.665003 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:27.664970    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:29.247241 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:29.247201    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:29.451323 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:29.451290    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:30.428465 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:30.428413    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:30.627349 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:30.627316    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:33.678939 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:33.678906    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:33.881538 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:33.881504    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:37.467886 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:37.467791    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:38.432245 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:38.432212    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:40.083003 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:11:40.082957    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:11:41.924787 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:41.924758    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"]
Nov 03 23:11:41.925173 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:41.924944    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd" podUID="11fe8f39-c007-4989-bdba-7ef33abe9647" containerName="maas-api" containerID="cri-o://c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9" gracePeriod=30
Nov 03 23:11:42.125000 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.124966    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:42.178979 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.178909    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 23:11:42.262233 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.262199    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11fe8f39-c007-4989-bdba-7ef33abe9647-kube-api-access-j9h95\" (UniqueName: \"kubernetes.io/projected/11fe8f39-c007-4989-bdba-7ef33abe9647-kube-api-access-j9h95\") pod \"11fe8f39-c007-4989-bdba-7ef33abe9647\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") "
Nov 03 23:11:42.262451 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.262251    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls\") pod \"11fe8f39-c007-4989-bdba-7ef33abe9647\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") "
Nov 03 23:11:42.262451 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.262289    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11fe8f39-c007-4989-bdba-7ef33abe9647-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fe8f39-c007-4989-bdba-7ef33abe9647-openshift-service-ca\") pod \"11fe8f39-c007-4989-bdba-7ef33abe9647\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") "
Nov 03 23:11:42.262451 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.262430    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-metrics-certs\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-metrics-certs\") pod \"11fe8f39-c007-4989-bdba-7ef33abe9647\" (UID: \"11fe8f39-c007-4989-bdba-7ef33abe9647\") "
Nov 03 23:11:42.262741 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.262706    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11fe8f39-c007-4989-bdba-7ef33abe9647-openshift-service-ca" pod "11fe8f39-c007-4989-bdba-7ef33abe9647" (UID: "11fe8f39-c007-4989-bdba-7ef33abe9647"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:11:42.262871 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.262794    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fe8f39-c007-4989-bdba-7ef33abe9647-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:42.264463 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.264439    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11fe8f39-c007-4989-bdba-7ef33abe9647-kube-api-access-j9h95" pod "11fe8f39-c007-4989-bdba-7ef33abe9647" (UID: "11fe8f39-c007-4989-bdba-7ef33abe9647"). InnerVolumeSpecName "kube-api-access-j9h95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:11:42.264463 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.264446    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls" pod "11fe8f39-c007-4989-bdba-7ef33abe9647" (UID: "11fe8f39-c007-4989-bdba-7ef33abe9647"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:11:42.264641 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.264502    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-metrics-certs" pod "11fe8f39-c007-4989-bdba-7ef33abe9647" (UID: "11fe8f39-c007-4989-bdba-7ef33abe9647"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:11:42.363992 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.363959    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:42.363992 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.363987    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9h95\" (UniqueName: \"kubernetes.io/projected/11fe8f39-c007-4989-bdba-7ef33abe9647-kube-api-access-j9h95\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:42.363992 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:42.364000    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11fe8f39-c007-4989-bdba-7ef33abe9647-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:43.006480 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.006445    5162 generic.go:356] "Generic (PLEG): container finished" podID="11fe8f39-c007-4989-bdba-7ef33abe9647" containerID="c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9" exitCode=0
Nov 03 23:11:43.006875 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.006496    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"
Nov 03 23:11:43.006875 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.006530    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd" event={"ID":"11fe8f39-c007-4989-bdba-7ef33abe9647","Type":"ContainerDied","Data":"c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9"}
Nov 03 23:11:43.006875 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.006585    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd" event={"ID":"11fe8f39-c007-4989-bdba-7ef33abe9647","Type":"ContainerDied","Data":"df808b4ef8b1fcf61aab5310e97f1a5f36038416ce82cbdce0ce2ab34c6cc7de"}
Nov 03 23:11:43.006875 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.006614    5162 scope.go:122] "RemoveContainer" containerID="c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9"
Nov 03 23:11:43.015259 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.015239    5162 scope.go:122] "RemoveContainer" containerID="c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9"
Nov 03 23:11:43.015546 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:11:43.015516    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9\": container with ID starting with c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9 not found: ID does not exist" containerID="c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9"
Nov 03 23:11:43.015626 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.015549    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9"} err="failed to get container status \"c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9\": rpc error: code = NotFound desc = could not find container \"c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9\": container with ID starting with c455068ea8f49a8c0a68221efd83ae26d37e11cd8a5c922bc745ec6826dfd0d9 not found: ID does not exist"
Nov 03 23:11:43.175727 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.175699    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"]
Nov 03 23:11:43.227127 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.227102    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-6f1190-64cb8fd5c5-l8bzd"]
Nov 03 23:11:43.459161 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.459077    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:43.660072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.660036    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:43.898035 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:43.897952    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:44.531585 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:44.531553    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:44.853173 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:44.853088    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11fe8f39-c007-4989-bdba-7ef33abe9647" path="/var/lib/kubelet/pods/11fe8f39-c007-4989-bdba-7ef33abe9647/volumes"
Nov 03 23:11:45.602369 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602334    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"]
Nov 03 23:11:45.602918 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602898    5162 cpu_manager.go:414] "containerMap: removing container" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602921    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602941    5162 cpu_manager.go:414] "containerMap: removing container" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602948    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602958    5162 cpu_manager.go:414] "containerMap: removing container" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602965    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602980    5162 cpu_manager.go:414] "containerMap: removing container" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602987    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.602998    5162 cpu_manager.go:414] "containerMap: removing container" podUID="abd9e7fd-4f46-48ba-9947-75a14210c989" containerName="maas-api"
Nov 03 23:11:45.603006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603005    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abd9e7fd-4f46-48ba-9947-75a14210c989" containerName="maas-api"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603021    5162 cpu_manager.go:414] "containerMap: removing container" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603028    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603058    5162 cpu_manager.go:414] "containerMap: removing container" podUID="46dcc5d8-b2d9-4c60-9e76-8cf25739094b" containerName="maas-api"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603064    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46dcc5d8-b2d9-4c60-9e76-8cf25739094b" containerName="maas-api"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603081    5162 cpu_manager.go:414] "containerMap: removing container" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603088    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603100    5162 cpu_manager.go:414] "containerMap: removing container" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603107    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603118    5162 cpu_manager.go:414] "containerMap: removing container" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603126    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603137    5162 cpu_manager.go:414] "containerMap: removing container" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603143    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603160    5162 cpu_manager.go:414] "containerMap: removing container" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603166    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603176    5162 cpu_manager.go:414] "containerMap: removing container" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603185    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603201    5162 cpu_manager.go:414] "containerMap: removing container" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603208    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603221    5162 cpu_manager.go:414] "containerMap: removing container" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603227    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603237    5162 cpu_manager.go:414] "containerMap: removing container" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603244    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603254    5162 cpu_manager.go:414] "containerMap: removing container" podUID="c3268422-a2de-48be-ba19-1860bead948c" containerName="discovery"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603261    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3268422-a2de-48be-ba19-1860bead948c" containerName="discovery"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603270    5162 cpu_manager.go:414] "containerMap: removing container" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603280    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603289    5162 cpu_manager.go:414] "containerMap: removing container" podUID="11fe8f39-c007-4989-bdba-7ef33abe9647" containerName="maas-api"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603296    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11fe8f39-c007-4989-bdba-7ef33abe9647" containerName="maas-api"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603405    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603416    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="c73f4e32-0faa-48a1-8953-eca91366445c" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603426    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9233afc-7975-4d25-b869-612a49d4038a" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603438    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="59552c85-6ea7-42f0-9b94-eb639f8f0dfd" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603447    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603458    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603466    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d28165a-2bf2-44b9-be71-d3cc7ed742e5" containerName="main"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603474    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2911c96-53cb-4128-8f79-0975e1b97894" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603480    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3268422-a2de-48be-ba19-1860bead948c" containerName="discovery"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603488    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="013e4f6a-2cf6-4d66-8580-77c5e2359f53" containerName="istio-proxy"
Nov 03 23:11:45.603479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603496    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="11fe8f39-c007-4989-bdba-7ef33abe9647" containerName="maas-api"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603504    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a193c2a-098b-49bd-9924-0b6b3fc70718" containerName="main"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603514    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="f923fc18-ec7e-4028-865c-c6e0f1b40f3f" containerName="main"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603523    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe0cad46-96b3-44fb-81a4-9415d4db7046" containerName="main"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603532    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="ede9ac93-5f42-48c8-85a0-848d53fa5056" containerName="main"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603540    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="abd9e7fd-4f46-48ba-9947-75a14210c989" containerName="maas-api"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603548    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="857d62dc-6b25-4d48-b706-d995c5fc2fef" containerName="istio-proxy"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603561    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="46dcc5d8-b2d9-4c60-9e76-8cf25739094b" containerName="maas-api"
Nov 03 23:11:45.605108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.603569    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="395a00c0-b9af-420d-9b3f-cf928586bff2" containerName="istio-proxy"
Nov 03 23:11:45.608581 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.608562    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.662513 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.662496    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-6f1190-openshift-default-dockercfg-fjmzm\""
Nov 03 23:11:45.696920 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.696897    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9vg4\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-kube-api-access-k9vg4\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697037 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.696974    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697089 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.697063    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697129 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.697104    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697129 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.697122    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697259 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.697243    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697318 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.697282    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697430 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.697410    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.697498 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.697482    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.717622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.717594    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"]
Nov 03 23:11:45.798091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798052    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9vg4\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-kube-api-access-k9vg4\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798247 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798128    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798276    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798314    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798501 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798430    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798501 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798475    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798618 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798505    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798618 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798556    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798719 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798628    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798719 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798703    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-data\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798823 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798772    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-certs\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798920 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798890    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-credential-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.798985 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.798931    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-socket\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.799262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.799239    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istiod-ca-cert\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.800568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.800549    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-podinfo\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.800643 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.800608    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-envoy\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.860746 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.860695    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-token\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.860871 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.860853    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9vg4\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-kube-api-access-k9vg4\") pod \"e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") " pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:45.920795 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:45.920773    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:46.337473 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:11:46.337435    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9bbd4a_41c9_4347_b7b8_5b604b93ba3c.slice/crio-b0ea539660e2eab8ce809a38a6a47fbefc672279c1308ac95313ce07cb9d5729 WatchSource:0}: Error finding container b0ea539660e2eab8ce809a38a6a47fbefc672279c1308ac95313ce07cb9d5729: Status 404 returned error can't find the container with id b0ea539660e2eab8ce809a38a6a47fbefc672279c1308ac95313ce07cb9d5729
Nov 03 23:11:46.339726 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:46.339691    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:11:46.339816 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:46.339761    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:11:46.339816 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:46.339790    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:11:46.339979 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:46.339953    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"]
Nov 03 23:11:47.029280 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:47.029244    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" event={"ID":"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c","Type":"ContainerStarted","Data":"f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0"}
Nov 03 23:11:47.029752 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:47.029300    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" event={"ID":"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c","Type":"ContainerStarted","Data":"b0ea539660e2eab8ce809a38a6a47fbefc672279c1308ac95313ce07cb9d5729"}
Nov 03 23:11:47.165459 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:47.158698    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" podStartSLOduration=2.158680245 podStartE2EDuration="2.158680245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:11:47.157909928 +0000 UTC m=+2910.422609967" watchObservedRunningTime="2026-11-03 23:11:47.158680245 +0000 UTC m=+2910.423380255"
Nov 03 23:11:47.921112 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:47.921078    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:47.922873 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:47.922844    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 23:11:47.922992 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:47.922900    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 23:11:48.922009 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:48.921978    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 23:11:48.922431 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:48.922040    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 23:11:49.921305 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:49.921259    5162 patch_prober.go:29] interesting pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 23:11:49.921482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:49.921312    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 23:11:50.435516 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:50.435486    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:50.638789 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:50.638756    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:50.722635 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:50.722555    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"]
Nov 03 23:11:50.722796 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:50.722726    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerName="istio-proxy" containerID="cri-o://f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0" gracePeriod=30
Nov 03 23:11:50.867044 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:50.867007    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:11:57.375134 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.375109    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:57.540895 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.540824    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-podinfo\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.540895 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.540891    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-credential-socket\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541076 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.540920    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-envoy\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541076 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.540982    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-socket\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541076 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541011    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-certs\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541213 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541073    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-token\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541213 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541121    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-kube-api-access-k9vg4\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-kube-api-access-k9vg4\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541213 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541161    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istiod-ca-cert\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541213 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541202    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-data\") pod \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\" (UID: \"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c\") "
Nov 03 23:11:57.541441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541222    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-credential-socket" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:57.541441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541307    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-socket" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:57.541553 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541470    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-certs" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:57.541663 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541633    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istiod-ca-cert" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:11:57.541795 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541667    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.541795 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541690    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.541795 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541704    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.541959 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.541926    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-data" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:57.543151 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.543125    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-podinfo" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:11:57.543151 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.543134    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-envoy" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:11:57.543532 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.543516    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-kube-api-access-k9vg4" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "kube-api-access-k9vg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:11:57.543642 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.543619    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-token" pod "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" (UID: "8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:11:57.642514 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.642482    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.642514 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.642509    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.642697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.642521    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.642697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.642535    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9vg4\" (UniqueName: \"kubernetes.io/projected/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-kube-api-access-k9vg4\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.642697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.642549    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:57.642697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:57.642562    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:11:58.107985 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.107955    5162 generic.go:356] "Generic (PLEG): container finished" podID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerID="f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0" exitCode=0
Nov 03 23:11:58.107985 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.107988    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" event={"ID":"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c","Type":"ContainerDied","Data":"f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0"}
Nov 03 23:11:58.108214 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.108002    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"
Nov 03 23:11:58.108214 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.108026    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr" event={"ID":"8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c","Type":"ContainerDied","Data":"b0ea539660e2eab8ce809a38a6a47fbefc672279c1308ac95313ce07cb9d5729"}
Nov 03 23:11:58.108214 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.108049    5162 scope.go:122] "RemoveContainer" containerID="f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0"
Nov 03 23:11:58.117539 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.117523    5162 scope.go:122] "RemoveContainer" containerID="f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0"
Nov 03 23:11:58.117775 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:11:58.117756    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0\": container with ID starting with f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0 not found: ID does not exist" containerID="f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0"
Nov 03 23:11:58.117815 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.117783    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0"} err="failed to get container status \"f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0\": rpc error: code = NotFound desc = could not find container \"f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0\": container with ID starting with f5ed9ba08b17601d0bea29206fa4f761db94b1d5489eaa17c15fe6cd7a0201e0 not found: ID does not exist"
Nov 03 23:11:58.330591 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.330553    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"]
Nov 03 23:11:58.382274 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.382197    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr"]
Nov 03 23:11:58.853482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:11:58.853449    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" path="/var/lib/kubelet/pods/8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c/volumes"
Nov 03 23:12:10.083846 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:12:10.083807    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:12:19.258857 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:19.258822    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"]
Nov 03 23:12:19.259329 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:19.259064    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy" containerID="cri-o://b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761" gracePeriod=30
Nov 03 23:12:24.149855 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.149821    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:12:24.504254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.504231    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 23:12:24.602003 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.601980    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-certs\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602151 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602011    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-credential-socket\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602151 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602066    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-token\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-token\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602151 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602098    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-kube-api-access-84zvk\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-kube-api-access-84zvk\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602151 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602127    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istiod-ca-cert\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602187    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-envoy\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602228    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-data\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602254    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-podinfo\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602289    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-certs" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:12:24.602385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602348    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-socket\") pod \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\" (UID: \"02b3b9d5-4ba8-44dd-9e78-b75025735de8\") "
Nov 03 23:12:24.602694 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602539    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-credential-socket" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:12:24.602694 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602555    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-data" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:12:24.602694 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602584    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istiod-ca-cert" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:12:24.602862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602736    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-socket" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:12:24.602862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602821    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.602862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602841    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.602862 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602853    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.603075 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602862    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.603075 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.602875    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.604434 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.604375    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-token" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:12:24.604434 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.604408    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-podinfo" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:12:24.604681 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.604659    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-envoy" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:12:24.604770 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.604710    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-kube-api-access-84zvk" pod "02b3b9d5-4ba8-44dd-9e78-b75025735de8" (UID: "02b3b9d5-4ba8-44dd-9e78-b75025735de8"). InnerVolumeSpecName "kube-api-access-84zvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:12:24.703886 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.703822    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.703886 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.703848    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.703886 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.703858    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:24.703886 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:24.703867    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84zvk\" (UniqueName: \"kubernetes.io/projected/02b3b9d5-4ba8-44dd-9e78-b75025735de8-kube-api-access-84zvk\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:12:25.251055 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.251022    5162 generic.go:356] "Generic (PLEG): container finished" podID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerID="b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761" exitCode=0
Nov 03 23:12:25.251497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.251067    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"
Nov 03 23:12:25.251497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.251074    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" event={"ID":"02b3b9d5-4ba8-44dd-9e78-b75025735de8","Type":"ContainerDied","Data":"b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761"}
Nov 03 23:12:25.251497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.251113    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm" event={"ID":"02b3b9d5-4ba8-44dd-9e78-b75025735de8","Type":"ContainerDied","Data":"08f340a29a3761ee67c203647ac767aac818c031dbde773bb7f4d107cc444c4b"}
Nov 03 23:12:25.251497 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.251133    5162 scope.go:122] "RemoveContainer" containerID="b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761"
Nov 03 23:12:25.259854 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.259839    5162 scope.go:122] "RemoveContainer" containerID="b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761"
Nov 03 23:12:25.260120 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:12:25.260099    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761\": container with ID starting with b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761 not found: ID does not exist" containerID="b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761"
Nov 03 23:12:25.260166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.260128    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761"} err="failed to get container status \"b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761\": rpc error: code = NotFound desc = could not find container \"b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761\": container with ID starting with b3585bce050990ba4cc3cacdabbca787c162df1f390ce794abf55c8191ba6761 not found: ID does not exist"
Nov 03 23:12:25.422322 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.422291    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"]
Nov 03 23:12:25.473880 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.473850    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm"]
Nov 03 23:12:25.548038 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:25.547967    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:12:26.853012 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:26.852983    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" path="/var/lib/kubelet/pods/02b3b9d5-4ba8-44dd-9e78-b75025735de8/volumes"
Nov 03 23:12:27.855054 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:27.855018    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:12:34.407685 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.407654    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"]
Nov 03 23:12:34.408073 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.408060    5162 cpu_manager.go:414] "containerMap: removing container" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy"
Nov 03 23:12:34.408120 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.408074    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy"
Nov 03 23:12:34.408120 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.408087    5162 cpu_manager.go:414] "containerMap: removing container" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerName="istio-proxy"
Nov 03 23:12:34.408120 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.408092    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerName="istio-proxy"
Nov 03 23:12:34.408214 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.408150    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="02b3b9d5-4ba8-44dd-9e78-b75025735de8" containerName="istio-proxy"
Nov 03 23:12:34.408214 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.408161    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b9bbd4a-41c9-4347-b7b8-5b604b93ba3c" containerName="istio-proxy"
Nov 03 23:12:34.411360 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.411345    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.487972 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.487939    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:12:34.491125 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491098    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491135    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491154    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491221    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491251    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491539 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491280    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h6gf\" (UniqueName: \"kubernetes.io/projected/4ddece53-145f-4433-b5ad-8aa6b6576873-kube-api-access-7h6gf\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491539 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491357    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddece53-145f-4433-b5ad-8aa6b6576873-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491539 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491377    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.491539 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.491523    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592189    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592245    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592265    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592302    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592327    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592368    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h6gf\" (UniqueName: \"kubernetes.io/projected/4ddece53-145f-4433-b5ad-8aa6b6576873-kube-api-access-7h6gf\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592415 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592412    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddece53-145f-4433-b5ad-8aa6b6576873-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592762 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592437    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592762 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592494    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592762 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592709    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.592975 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592939    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.593045 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.592999    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.593108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.593086    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.593295 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.593273    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddece53-145f-4433-b5ad-8aa6b6576873-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.594848 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.594825    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.595162 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.595141    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.623314 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.623276    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"]
Nov 03 23:12:34.654750 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.654722    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddece53-145f-4433-b5ad-8aa6b6576873-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.654944 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.654923    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h6gf\" (UniqueName: \"kubernetes.io/projected/4ddece53-145f-4433-b5ad-8aa6b6576873-kube-api-access-7h6gf\") pod \"maas-default-gateway-openshift-default-59f957fdd7-chpk6\" (UID: \"4ddece53-145f-4433-b5ad-8aa6b6576873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:34.724073 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:34.724030    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:35.143454 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:12:35.143416    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ddece53_145f_4433_b5ad_8aa6b6576873.slice/crio-febd89f6dfa7ec07c2660f41c30f78de4f80109496db4b472e2f9a69c0cb59c8 WatchSource:0}: Error finding container febd89f6dfa7ec07c2660f41c30f78de4f80109496db4b472e2f9a69c0cb59c8: Status 404 returned error can't find the container with id febd89f6dfa7ec07c2660f41c30f78de4f80109496db4b472e2f9a69c0cb59c8
Nov 03 23:12:35.145609 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.145576    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:12:35.145696 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.145648    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:12:35.145696 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.145679    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:12:35.145890 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.145870    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"]
Nov 03 23:12:35.304435 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.304383    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" event={"ID":"4ddece53-145f-4433-b5ad-8aa6b6576873","Type":"ContainerStarted","Data":"30db4f4a20606326890ccc81037e82a83ab09d478f578a6484545c93acd4d202"}
Nov 03 23:12:35.304605 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.304445    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" event={"ID":"4ddece53-145f-4433-b5ad-8aa6b6576873","Type":"ContainerStarted","Data":"febd89f6dfa7ec07c2660f41c30f78de4f80109496db4b472e2f9a69c0cb59c8"}
Nov 03 23:12:35.429824 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.429774    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podStartSLOduration=1.42976031 podStartE2EDuration="1.42976031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:12:35.4282516 +0000 UTC m=+2958.692951608" watchObservedRunningTime="2026-11-03 23:12:35.42976031 +0000 UTC m=+2958.694460348"
Nov 03 23:12:35.724400 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:35.724309    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:36.726241 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:36.726202    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 23:12:36.726635 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:36.726252    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podUID="4ddece53-145f-4433-b5ad-8aa6b6576873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 23:12:37.724614 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:37.724581    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 23:12:37.724791 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:37.724632    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podUID="4ddece53-145f-4433-b5ad-8aa6b6576873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 23:12:38.724930 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:38.724901    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 23:12:38.725289 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:38.724950    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podUID="4ddece53-145f-4433-b5ad-8aa6b6576873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 23:12:39.725293 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:39.725259    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 23:12:39.725806 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:39.725322    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podUID="4ddece53-145f-4433-b5ad-8aa6b6576873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 23:12:40.089107 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:12:40.089015    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:12:40.724676 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:40.724639    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 23:12:40.724854 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:40.724693    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podUID="4ddece53-145f-4433-b5ad-8aa6b6576873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 23:12:41.707969 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:41.707933    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:12:41.725020 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:41.724996    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 23:12:41.725126 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:41.725038    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podUID="4ddece53-145f-4433-b5ad-8aa6b6576873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 23:12:43.140262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:43.140225    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:43.140710 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:43.140665    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:43.141487 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:43.141466    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6"
Nov 03 23:12:43.821699 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:43.821666    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:12:44.089995 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:44.089921    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:12:49.504693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.504656    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"]
Nov 03 23:12:49.508701 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.508675    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562139 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562110    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hgb8\" (UniqueName: \"kubernetes.io/projected/fc872ba5-e18e-4687-81f9-bcbb760f8faa-kube-api-access-7hgb8\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562139 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562141    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562340 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562170    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562340 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562284    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562340 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562323    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562553 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562377    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562553 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562460    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562553 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562492    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.562711 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.562625    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.617102 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.617076    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"]
Nov 03 23:12:49.666734 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666708    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.666876 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666741    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.666876 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666768    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.666876 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666789    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.666876 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666848    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666887    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hgb8\" (UniqueName: \"kubernetes.io/projected/fc872ba5-e18e-4687-81f9-bcbb760f8faa-kube-api-access-7hgb8\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666912    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.666944    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.667033    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.667171    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-workload-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.667233    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-data\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667454 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.667433    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-credential-socket\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667530 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.667479    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-workload-certs\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.667593 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.667573    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.669114 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.669095    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-envoy\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.669249 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.669229    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.729302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.729278    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc872ba5-e18e-4687-81f9-bcbb760f8faa-istio-token\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.729521 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.729503    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hgb8\" (UniqueName: \"kubernetes.io/projected/fc872ba5-e18e-4687-81f9-bcbb760f8faa-kube-api-access-7hgb8\") pod \"maas-default-gateway-openshift-default-59f957fdd7-66p28\" (UID: \"fc872ba5-e18e-4687-81f9-bcbb760f8faa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:49.822419 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:49.822345    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:50.239692 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:12:50.239658    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc872ba5_e18e_4687_81f9_bcbb760f8faa.slice/crio-d6bda0fb0fd7aeb3ce0127480e70dd7adb1137a749148d698a231a0e9b32ab0f WatchSource:0}: Error finding container d6bda0fb0fd7aeb3ce0127480e70dd7adb1137a749148d698a231a0e9b32ab0f: Status 404 returned error can't find the container with id d6bda0fb0fd7aeb3ce0127480e70dd7adb1137a749148d698a231a0e9b32ab0f
Nov 03 23:12:50.242010 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.241975    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:12:50.242088 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.242041    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:12:50.242088 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.242067    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:12:50.242512 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.242455    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"]
Nov 03 23:12:50.384660 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.384622    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" event={"ID":"fc872ba5-e18e-4687-81f9-bcbb760f8faa","Type":"ContainerStarted","Data":"d88606ba8d2260d63131530183263246d6194970e6c67ab0c92967eef3e1d066"}
Nov 03 23:12:50.384851 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.384684    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" event={"ID":"fc872ba5-e18e-4687-81f9-bcbb760f8faa","Type":"ContainerStarted","Data":"d6bda0fb0fd7aeb3ce0127480e70dd7adb1137a749148d698a231a0e9b32ab0f"}
Nov 03 23:12:50.508728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.508637    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podStartSLOduration=1.508623896 podStartE2EDuration="1.508623896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:12:50.507125709 +0000 UTC m=+2973.771825718" watchObservedRunningTime="2026-11-03 23:12:50.508623896 +0000 UTC m=+2973.773323906"
Nov 03 23:12:50.823093 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:50.823003    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:51.824869 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:51.824836    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Nov 03 23:12:51.825265 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:51.824885    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podUID="fc872ba5-e18e-4687-81f9-bcbb760f8faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Nov 03 23:12:52.822881 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:52.822845    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Nov 03 23:12:52.823032 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:52.822897    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podUID="fc872ba5-e18e-4687-81f9-bcbb760f8faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Nov 03 23:12:53.823575 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:53.823537    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Nov 03 23:12:53.824086 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:53.823613    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podUID="fc872ba5-e18e-4687-81f9-bcbb760f8faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Nov 03 23:12:54.823335 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:54.823299    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Nov 03 23:12:54.823534 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:54.823349    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podUID="fc872ba5-e18e-4687-81f9-bcbb760f8faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Nov 03 23:12:55.823283 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:55.823249    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Nov 03 23:12:55.823724 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:55.823314    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podUID="fc872ba5-e18e-4687-81f9-bcbb760f8faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Nov 03 23:12:56.823412 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:56.823356    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Nov 03 23:12:56.823788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:56.823448    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podUID="fc872ba5-e18e-4687-81f9-bcbb760f8faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Nov 03 23:12:58.219803 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:58.219770    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:58.220442 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:58.220419    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:12:58.221257 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:12:58.221239    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28"
Nov 03 23:13:10.083911 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:13:10.083830    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:13:24.493522 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:13:24.493490    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:13:25.431600 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:13:25.431565    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:13:34.862414 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:13:34.859037    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:13:40.082542 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:13:40.082501    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:13:44.173953 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:13:44.173907    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:13:57.062609 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:13:57.062573    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:14:10.083976 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:14:10.083940    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:14:24.871156 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:14:24.871124    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:14:26.555235 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:14:26.555197    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:14:35.199753 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:14:35.199714    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:14:40.082844 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:14:40.082762    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:14:44.590444 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:14:44.590331    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:15:10.083096 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:15:10.083055    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:15:25.214329 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:15:25.214291    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:15:35.555327 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:15:35.555291    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:15:40.082777 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:15:40.082739    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:15:44.944106 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:15:44.944068    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:16:08.383471 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:16:08.383068    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:16:10.082637 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:16:10.082596    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:16:28.430056 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:16:28.430019    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:16:28.970113 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:16:28.970078    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:16:35.901170 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:16:35.901138    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:16:40.130267 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:16:40.130231    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:16:45.313405 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:16:45.313365    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:17:10.083113 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:17:10.083070    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:17:25.913193 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:17:25.913158    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:17:36.261308 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:17:36.261268    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:17:40.133565 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:17:40.133220    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:17:41.198424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:17:41.197506    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:17:41.198424 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:17:41.197576    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr" podUID="e2eb0dd5-b5d0-40b6-a00b-7761d4adadda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:17:45.605971 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:17:45.605934    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:18:10.083728 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:18:10.083692    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:18:14.465539 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:18:14.465505    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:18:26.287456 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:18:26.287417    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:18:36.603311 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:18:36.603270    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:18:40.133744 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:18:40.133704    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:19:10.083585 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:19:10.083501    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:19:26.695227 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:19:26.695188    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:19:36.969163 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:19:36.969121    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:19:40.083059 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:19:40.083021    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:19:51.028872 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:19:51.028834    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:19:58.387442 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:19:58.387374    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:07.763852 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:07.763817    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:08.202294 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.202266    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"]
Nov 03 23:20:08.205757 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.205742    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.259797 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.259768    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-1376dc-openshift-default-dockercfg-8dnxw\""
Nov 03 23:20:08.275245 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275206    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-certs\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275263    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-envoy\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275292    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istiod-ca-cert\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275313    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-socket\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275341    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-data\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275360    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgb2j\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-kube-api-access-jgb2j\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275425    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-token\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275670 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275613    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-podinfo\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.275670 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.275643    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-credential-socket\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.313458 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.313429    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"]
Nov 03 23:20:08.354504 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.354460    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:08.377109 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377076    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-envoy\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377117    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istiod-ca-cert\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377141    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-socket\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377168    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-data\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377184    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgb2j\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-kube-api-access-jgb2j\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377215    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-token\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377263    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-podinfo\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377292    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-credential-socket\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377656 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377351    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-certs\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377754 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377651    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-data\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377874    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istiod-ca-cert\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.377993 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377975    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-certs\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.378043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.377988    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-credential-socket\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.378089 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.378066    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-socket\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.379426 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.379406    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-envoy\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.379747 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.379726    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-podinfo\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.439900 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.439874    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-token\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.440192 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.440169    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgb2j\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-kube-api-access-jgb2j\") pod \"e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") " pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.518061 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.517990    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:08.717119 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.717083    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:08.937464 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:20:08.937429    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c3dedf_73e4_4fe5_954b_31b57a41ab9f.slice/crio-6e6e31087727228b2c177c2fc2baa7a3b6670a6c31b14ce795149029bbdef2aa WatchSource:0}: Error finding container 6e6e31087727228b2c177c2fc2baa7a3b6670a6c31b14ce795149029bbdef2aa: Status 404 returned error can't find the container with id 6e6e31087727228b2c177c2fc2baa7a3b6670a6c31b14ce795149029bbdef2aa
Nov 03 23:20:08.939641 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.939609    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:20:08.939752 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.939691    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:20:08.939752 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.939733    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:20:08.939861 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:08.939845    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"]
Nov 03 23:20:09.811556 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:09.811516    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" event={"ID":"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f","Type":"ContainerStarted","Data":"0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093"}
Nov 03 23:20:09.811732 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:09.811575    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" event={"ID":"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f","Type":"ContainerStarted","Data":"6e6e31087727228b2c177c2fc2baa7a3b6670a6c31b14ce795149029bbdef2aa"}
Nov 03 23:20:09.988027 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:09.987976    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" podStartSLOduration=1.9879629840000002 podStartE2EDuration="1.987962984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:20:09.986686731 +0000 UTC m=+3413.251386740" watchObservedRunningTime="2026-11-03 23:20:09.987962984 +0000 UTC m=+3413.252662992"
Nov 03 23:20:10.082638 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:20:10.082562    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:20:10.519048 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:10.519015    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:10.524179 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:10.524157    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:10.816368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:10.816291    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:10.817215 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:10.817199    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:12.896652 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:12.896620    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:13.102979 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:13.102946    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:13.346376 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:13.346339    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:16.169661 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:16.169630    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:16.375643 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:16.375599    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:18.817966 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.817926    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"]
Nov 03 23:20:18.821969 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.821947    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:18.877518 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.877486    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"]
Nov 03 23:20:18.926248 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.926218    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-1376dc-serving-cert\""
Nov 03 23:20:18.983960 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.983918    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjxvg\" (UniqueName: \"kubernetes.io/projected/6f8dbe70-036c-439f-b6f7-553ed94f88c6-kube-api-access-qjxvg\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:18.984138 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.983978    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:18.984138 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.984005    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:18.984492 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:18.984464    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8dbe70-036c-439f-b6f7-553ed94f88c6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.081928 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.081850    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:19.085947 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.085918    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjxvg\" (UniqueName: \"kubernetes.io/projected/6f8dbe70-036c-439f-b6f7-553ed94f88c6-kube-api-access-qjxvg\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.086111 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.085971    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.086111 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.086078    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.086240 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.086171    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8dbe70-036c-439f-b6f7-553ed94f88c6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.086240 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:20:19.086213    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-serving-cert: secret "maas-api-e2e-worker-single-main-1376dc-serving-cert" not found
Nov 03 23:20:19.086332 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:20:19.086297    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls podName:6f8dbe70-036c-439f-b6f7-553ed94f88c6 nodeName:}" failed. No retries permitted until 2026-11-03 23:20:19.586276828 +0000 UTC m=+3422.850976817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls") pod "maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl" (UID: "6f8dbe70-036c-439f-b6f7-553ed94f88c6") : secret "maas-api-e2e-worker-single-main-1376dc-serving-cert" not found
Nov 03 23:20:19.086879 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.086861    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8dbe70-036c-439f-b6f7-553ed94f88c6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.089120 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.089089    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.150778 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.150751    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjxvg\" (UniqueName: \"kubernetes.io/projected/6f8dbe70-036c-439f-b6f7-553ed94f88c6-kube-api-access-qjxvg\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.287183 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.287147    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:19.489425 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.489372    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:19.590256 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.590223    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.594709 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.594683    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:19.734121 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:19.734083    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:20.107637 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:20:20.107606    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f8dbe70_036c_439f_b6f7_553ed94f88c6.slice/crio-8bfd1232962dbec1ed289564d843fb3c1535e0116bf8913bf4377ab7a6bb88be WatchSource:0}: Error finding container 8bfd1232962dbec1ed289564d843fb3c1535e0116bf8913bf4377ab7a6bb88be: Status 404 returned error can't find the container with id 8bfd1232962dbec1ed289564d843fb3c1535e0116bf8913bf4377ab7a6bb88be
Nov 03 23:20:20.109350 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:20.109330    5162 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 23:20:20.110223 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:20.110135    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"]
Nov 03 23:20:20.865883 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:20.865848    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl" event={"ID":"6f8dbe70-036c-439f-b6f7-553ed94f88c6","Type":"ContainerStarted","Data":"8bfd1232962dbec1ed289564d843fb3c1535e0116bf8913bf4377ab7a6bb88be"}
Nov 03 23:20:21.872042 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:21.871989    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl" event={"ID":"6f8dbe70-036c-439f-b6f7-553ed94f88c6","Type":"ContainerStarted","Data":"769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8"}
Nov 03 23:20:21.872533 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:21.872066    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:22.011379 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:22.011316    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl" podStartSLOduration=3.057582873 podStartE2EDuration="4.01130318s" totalImagesPullingTime="953.720307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:20:22.009281629 +0000 UTC m=+3425.273981640" watchObservedRunningTime="2026-11-03 23:20:22.01130318 +0000 UTC m=+3425.276003188"
Nov 03 23:20:24.021968 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:24.021928    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:26.986098 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:26.986063    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:27.881965 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:27.881938    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:20:28.910840 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:28.910798    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:33.741495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:33.741456    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:35.945818 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:35.945790    5162 generic.go:356] "Generic (PLEG): container finished" podID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerID="0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093" exitCode=0
Nov 03 23:20:35.946172 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:35.945858    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" event={"ID":"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f","Type":"ContainerDied","Data":"0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093"}
Nov 03 23:20:35.946236 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:35.946221    5162 scope.go:122] "RemoveContainer" containerID="0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093"
Nov 03 23:20:35.947403 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:35.947357    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:20:35.947475 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:35.947450    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:20:35.947513 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:35.947483    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:20:36.971863 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:36.971822    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" event={"ID":"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f","Type":"ContainerStarted","Data":"03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92"}
Nov 03 23:20:37.310568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:37.310457    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:37.518254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:37.518223    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:37.519614 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:37.519586    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 23:20:37.519728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:37.519647    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 23:20:37.704659 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:37.704624    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:38.297131 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:38.297096    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:38.518363 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:38.518327    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 23:20:38.518555 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:38.518405    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 23:20:38.571619 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:38.571541    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:38.897906 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:38.897812    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:39.519125 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:39.519089    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 23:20:39.519561 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:39.519156    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 23:20:40.089935 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:20:40.089898    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:20:40.519112 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:40.519076    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 23:20:40.519325 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:40.519135    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 23:20:40.816618 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:40.816531    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:41.160470 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:41.156004    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:41.519328 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:41.519289    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 23:20:41.519761 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:41.519359    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 23:20:42.522722 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:42.522691    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:42.523699 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:42.523677    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:20:43.643684 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:43.643651    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:48.427239 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:48.427194    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:51.362313 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:51.362271    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:53.215770 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:53.215738    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:20:58.111820 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:20:58.111761    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:02.910380 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:02.910337    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:07.795337 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:07.795301    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:10.083689 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:21:10.083651    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:21:12.611313 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:12.611281    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:17.438495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:17.438454    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:22.211142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:22.211095    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:27.108140 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:27.108100    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:27.340547 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:27.340509    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:31.421384 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:31.421348    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:31.908201 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:31.908102    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:36.737269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:36.737236    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:37.670440 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:37.670377    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:40.083566 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:21:40.083529    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:21:41.512712 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:41.512661    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:46.329349 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:46.329286    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:49.628269 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:49.628236    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:51.344000 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:51.343953    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:51.708856 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:51.708817    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:54.729675 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:54.729638    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:21:56.519517 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:21:56.519480    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:01.316172 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:01.316097    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:02.053462 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:02.053428    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:02.292645 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:02.292607    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:05.386568 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:05.386530    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"]
Nov 03 23:22:05.387059 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:05.386806    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl" podUID="6f8dbe70-036c-439f-b6f7-553ed94f88c6" containerName="maas-api" containerID="cri-o://769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8" gracePeriod=30
Nov 03 23:22:05.544630 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:05.544595    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:06.644179 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.644153    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:22:06.763662 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.763597    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f8dbe70-036c-439f-b6f7-553ed94f88c6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8dbe70-036c-439f-b6f7-553ed94f88c6-openshift-service-ca\") pod \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") "
Nov 03 23:22:06.763802 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.763675    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-metrics-certs\") pod \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") "
Nov 03 23:22:06.763802 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.763751    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f8dbe70-036c-439f-b6f7-553ed94f88c6-kube-api-access-qjxvg\" (UniqueName: \"kubernetes.io/projected/6f8dbe70-036c-439f-b6f7-553ed94f88c6-kube-api-access-qjxvg\") pod \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") "
Nov 03 23:22:06.763905 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.763810    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls\") pod \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\" (UID: \"6f8dbe70-036c-439f-b6f7-553ed94f88c6\") "
Nov 03 23:22:06.764180 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.764118    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f8dbe70-036c-439f-b6f7-553ed94f88c6-openshift-service-ca" pod "6f8dbe70-036c-439f-b6f7-553ed94f88c6" (UID: "6f8dbe70-036c-439f-b6f7-553ed94f88c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:22:06.768085 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.767898    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f8dbe70-036c-439f-b6f7-553ed94f88c6-kube-api-access-qjxvg" pod "6f8dbe70-036c-439f-b6f7-553ed94f88c6" (UID: "6f8dbe70-036c-439f-b6f7-553ed94f88c6"). InnerVolumeSpecName "kube-api-access-qjxvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:22:06.768085 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.768007    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls" pod "6f8dbe70-036c-439f-b6f7-553ed94f88c6" (UID: "6f8dbe70-036c-439f-b6f7-553ed94f88c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:22:06.768085 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.768049    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-metrics-certs" pod "6f8dbe70-036c-439f-b6f7-553ed94f88c6" (UID: "6f8dbe70-036c-439f-b6f7-553ed94f88c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:22:06.865152 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.865128    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:06.865152 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.865153    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f8dbe70-036c-439f-b6f7-553ed94f88c6-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:06.865307 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.865162    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f8dbe70-036c-439f-b6f7-553ed94f88c6-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:06.865307 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:06.865170    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjxvg\" (UniqueName: \"kubernetes.io/projected/6f8dbe70-036c-439f-b6f7-553ed94f88c6-kube-api-access-qjxvg\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:07.081825 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.081744    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:07.360679 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.360564    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:07.458760 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.458729    5162 generic.go:356] "Generic (PLEG): container finished" podID="6f8dbe70-036c-439f-b6f7-553ed94f88c6" containerID="769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8" exitCode=0
Nov 03 23:22:07.458942 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.458775    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"
Nov 03 23:22:07.458942 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.458790    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl" event={"ID":"6f8dbe70-036c-439f-b6f7-553ed94f88c6","Type":"ContainerDied","Data":"769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8"}
Nov 03 23:22:07.458942 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.458833    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl" event={"ID":"6f8dbe70-036c-439f-b6f7-553ed94f88c6","Type":"ContainerDied","Data":"8bfd1232962dbec1ed289564d843fb3c1535e0116bf8913bf4377ab7a6bb88be"}
Nov 03 23:22:07.458942 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.458852    5162 scope.go:122] "RemoveContainer" containerID="769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8"
Nov 03 23:22:07.468180 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.468164    5162 scope.go:122] "RemoveContainer" containerID="769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8"
Nov 03 23:22:07.468503 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:22:07.468482    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8\": container with ID starting with 769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8 not found: ID does not exist" containerID="769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8"
Nov 03 23:22:07.468571 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.468512    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8"} err="failed to get container status \"769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8\": rpc error: code = NotFound desc = could not find container \"769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8\": container with ID starting with 769f99ce069054e2bfe9f943bf71bb90d891688feaab3adc13a66c1bfefe1bb8 not found: ID does not exist"
Nov 03 23:22:07.628184 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.628108    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"]
Nov 03 23:22:07.631916 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:07.631895    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-1376dc-795c6df786-prnxl"]
Nov 03 23:22:08.139924 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:08.139890    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:08.853322 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:08.853294    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f8dbe70-036c-439f-b6f7-553ed94f88c6" path="/var/lib/kubelet/pods/6f8dbe70-036c-439f-b6f7-553ed94f88c6/volumes"
Nov 03 23:22:10.083230 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:22:10.083161    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:22:14.435634 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:14.435590    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:14.712935 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:14.712853    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:14.714487 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:14.714468    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"]
Nov 03 23:22:14.714707 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:14.714670    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy" containerID="cri-o://03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92" gracePeriod=30
Nov 03 23:22:14.925456 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:14.925421    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:19.953603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:19.953579    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:22:20.098072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098002    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-token\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-token\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098039    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-credential-socket\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098300 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098094    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istiod-ca-cert\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098300 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098123    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-data\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098300 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098162    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-socket\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098300 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098216    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-certs\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098300 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098272    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-kube-api-access-jgb2j\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-kube-api-access-jgb2j\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098323    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-podinfo\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098373    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-envoy\") pod \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\" (UID: \"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f\") "
Nov 03 23:22:20.098607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098421    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-credential-socket" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:22:20.098607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098497    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-data" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:22:20.098607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098516    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-socket" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:22:20.098607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098538    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-certs" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:22:20.098607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098548    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istiod-ca-cert" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:22:20.098943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098797    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.098943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098821    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.098943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098835    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.098943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098849    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.098943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.098862    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.100438 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.100402    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-token" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:22:20.100438 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.100425    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-kube-api-access-jgb2j" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "kube-api-access-jgb2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:22:20.100589 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.100496    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-podinfo" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:22:20.100589 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.100517    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-envoy" pod "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" (UID: "e8c3dedf-73e4-4fe5-954b-31b57a41ab9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:22:20.199857 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.199831    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgb2j\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-kube-api-access-jgb2j\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.199857 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.199854    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.199857 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.199865    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.200074 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.199874    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:22:20.534079 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.534045    5162 generic.go:356] "Generic (PLEG): container finished" podID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerID="03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92" exitCode=0
Nov 03 23:22:20.534258 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.534086    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"
Nov 03 23:22:20.534258 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.534110    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" event={"ID":"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f","Type":"ContainerDied","Data":"03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92"}
Nov 03 23:22:20.534258 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.534157    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g" event={"ID":"e8c3dedf-73e4-4fe5-954b-31b57a41ab9f","Type":"ContainerDied","Data":"6e6e31087727228b2c177c2fc2baa7a3b6670a6c31b14ce795149029bbdef2aa"}
Nov 03 23:22:20.534258 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.534184    5162 scope.go:122] "RemoveContainer" containerID="03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92"
Nov 03 23:22:20.543683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.543668    5162 scope.go:122] "RemoveContainer" containerID="0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093"
Nov 03 23:22:20.551512 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.551495    5162 scope.go:122] "RemoveContainer" containerID="03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92"
Nov 03 23:22:20.551764 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:22:20.551744    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92\": container with ID starting with 03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92 not found: ID does not exist" containerID="03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92"
Nov 03 23:22:20.551845 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.551777    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92"} err="failed to get container status \"03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92\": rpc error: code = NotFound desc = could not find container \"03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92\": container with ID starting with 03a39fb3525ef55f6e714abaf035d98947de8dba000450492dae56a7dd7b9e92 not found: ID does not exist"
Nov 03 23:22:20.551845 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.551802    5162 scope.go:122] "RemoveContainer" containerID="0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093"
Nov 03 23:22:20.552033 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:22:20.552016    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093\": container with ID starting with 0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093 not found: ID does not exist" containerID="0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093"
Nov 03 23:22:20.552080 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.552041    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093"} err="failed to get container status \"0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093\": rpc error: code = NotFound desc = could not find container \"0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093\": container with ID starting with 0142a47393182ad99fb0d4cacd9f8c35fcacc347bfc22c07f93ece5bbafc5093 not found: ID does not exist"
Nov 03 23:22:20.704491 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.704459    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"]
Nov 03 23:22:20.708148 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.708127    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g"]
Nov 03 23:22:20.854085 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:20.854010    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" path="/var/lib/kubelet/pods/e8c3dedf-73e4-4fe5-954b-31b57a41ab9f/volumes"
Nov 03 23:22:27.705668 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:27.705631    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:30.714376 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:30.714327    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:31.596947 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:31.596894    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:37.345671 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:37.345635    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:38.034938 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:38.034889    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:40.082983 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:22:40.082948    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:22:51.990739 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:51.990703    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:56.198758 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:56.198714    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:22:56.402994 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:22:56.402960    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:23:10.083323 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:23:10.083289    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:23:28.047546 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:23:28.047511    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:23:40.082602 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:23:40.082520    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:23:48.387220 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:23:48.387172    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:23:52.388892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:23:52.388850    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:24:10.082897 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:24:10.082858    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:24:38.438067 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:24:38.438025    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:24:40.082459 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:24:40.082427    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:24:52.717925 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:24:52.717890    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:04.162649 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.162613    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:04.598473 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.598435    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"]
Nov 03 23:25:04.598845 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.598828    5162 cpu_manager.go:414] "containerMap: removing container" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy"
Nov 03 23:25:04.598930 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.598847    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy"
Nov 03 23:25:04.598930 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.598878    5162 cpu_manager.go:414] "containerMap: removing container" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy"
Nov 03 23:25:04.598930 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.598886    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy"
Nov 03 23:25:04.598930 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.598928    5162 cpu_manager.go:414] "containerMap: removing container" podUID="6f8dbe70-036c-439f-b6f7-553ed94f88c6" containerName="maas-api"
Nov 03 23:25:04.599078 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.598937    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f8dbe70-036c-439f-b6f7-553ed94f88c6" containerName="maas-api"
Nov 03 23:25:04.599078 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.599016    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f8dbe70-036c-439f-b6f7-553ed94f88c6" containerName="maas-api"
Nov 03 23:25:04.599078 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.599027    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy"
Nov 03 23:25:04.599078 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.599035    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8c3dedf-73e4-4fe5-954b-31b57a41ab9f" containerName="istio-proxy"
Nov 03 23:25:04.602169 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.602140    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.659609 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.659576    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-e433fd-openshift-default-dockercfg-7q42l\""
Nov 03 23:25:04.660448 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660419    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9g7r\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-kube-api-access-s9g7r\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660602 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660461    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f801e00-641b-4b27-a7af-9faed4f1235f-istiod-ca-cert\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660602 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660498    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-token\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660602 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660526    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-socket\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660602 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660561    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-envoy\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660609    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-data\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660666    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-credential-socket\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660713    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-certs\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.660896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.660794    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-podinfo\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.708871 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.708842    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"]
Nov 03 23:25:04.761493 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761458    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-envoy\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761513    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-data\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761550    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-credential-socket\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761580    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-certs\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761657    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-podinfo\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761833 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761709    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9g7r\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-kube-api-access-s9g7r\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761833 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761733    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f801e00-641b-4b27-a7af-9faed4f1235f-istiod-ca-cert\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761833 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761775    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-token\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.761833 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761801    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-socket\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.762024 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.761948    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-data\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.762083 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.762023    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-certs\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.762146 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.762087    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-socket\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.762260 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.762240    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-credential-socket\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.762573 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.762550    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f801e00-641b-4b27-a7af-9faed4f1235f-istiod-ca-cert\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.763912 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.763893    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-envoy\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.764193 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.764174    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-podinfo\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.775822 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.775792    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:04.820809 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.820782    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-token\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.821081 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.821057    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9g7r\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-kube-api-access-s9g7r\") pod \"e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") " pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:04.915443 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:04.915337    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:05.020348 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.020310    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:05.338676 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.338652    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"]
Nov 03 23:25:05.340434 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:25:05.340410    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f801e00_641b_4b27_a7af_9faed4f1235f.slice/crio-7b03099a1fb74ea5f9e13c374a177dff16ba54a2aec8afd4c931688de7cd679b WatchSource:0}: Error finding container 7b03099a1fb74ea5f9e13c374a177dff16ba54a2aec8afd4c931688de7cd679b: Status 404 returned error can't find the container with id 7b03099a1fb74ea5f9e13c374a177dff16ba54a2aec8afd4c931688de7cd679b
Nov 03 23:25:05.342641 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.342609    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:25:05.342736 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.342678    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:25:05.342736 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.342710    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:25:05.409351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.409318    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" event={"ID":"0f801e00-641b-4b27-a7af-9faed4f1235f","Type":"ContainerStarted","Data":"b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1"}
Nov 03 23:25:05.409511 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.409379    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" event={"ID":"0f801e00-641b-4b27-a7af-9faed4f1235f","Type":"ContainerStarted","Data":"7b03099a1fb74ea5f9e13c374a177dff16ba54a2aec8afd4c931688de7cd679b"}
Nov 03 23:25:05.527235 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.527174    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podStartSLOduration=1.527161483 podStartE2EDuration="1.527161483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:25:05.525265753 +0000 UTC m=+3708.789965765" watchObservedRunningTime="2026-11-03 23:25:05.527161483 +0000 UTC m=+3708.791861493"
Nov 03 23:25:05.916106 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:05.916073    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:06.921633 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:06.921602    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:07.420585 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:07.420556    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:07.421445 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:07.421424    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:08.439217 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:08.439184    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:08.695043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:08.694971    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:10.082168 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:25:10.082122    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:25:11.673757 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:11.673727    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:11.878838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:11.878797    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:14.612993 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.612961    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"]
Nov 03 23:25:14.616750 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.616729    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.660292 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.660263    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-metrics-certs\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.660441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.660308    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf42n\" (UniqueName: \"kubernetes.io/projected/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-kube-api-access-sf42n\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.660441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.660380    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.660558 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.660530    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.667155 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.667136    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-e433fd-serving-cert\""
Nov 03 23:25:14.717174 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.717143    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"]
Nov 03 23:25:14.761287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.761256    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-metrics-certs\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.761470 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.761306    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf42n\" (UniqueName: \"kubernetes.io/projected/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-kube-api-access-sf42n\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.761470 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.761338    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.761470 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.761453    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.761661 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:25:14.761610    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-serving-cert: secret "maas-api-e2e-worker-single-main-e433fd-serving-cert" not found
Nov 03 23:25:14.761716 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:25:14.761689    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls podName:4044f97c-7a5a-4b5e-a2dd-e17bfe91791c nodeName:}" failed. No retries permitted until 2026-11-03 23:25:15.261667182 +0000 UTC m=+3718.526367185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls") pod "maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" (UID: "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c") : secret "maas-api-e2e-worker-single-main-e433fd-serving-cert" not found
Nov 03 23:25:14.762025 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.762006    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.763621 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.763596    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-metrics-certs\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.822487 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.822459    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf42n\" (UniqueName: \"kubernetes.io/projected/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-kube-api-access-sf42n\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:14.875612 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:14.875525    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:15.096928 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:15.096865    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:15.267077 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:15.267047    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:15.269454 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:15.269425    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:15.316771 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:15.316738    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:15.530173 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:15.530084    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:15.897861 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:15.897837    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"]
Nov 03 23:25:15.899715 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:25:15.899689    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4044f97c_7a5a_4b5e_a2dd_e17bfe91791c.slice/crio-3992ca097ce85ee07c2580a14d3090c791ab52aaabf956a4470786e80a2a789e WatchSource:0}: Error finding container 3992ca097ce85ee07c2580a14d3090c791ab52aaabf956a4470786e80a2a789e: Status 404 returned error can't find the container with id 3992ca097ce85ee07c2580a14d3090c791ab52aaabf956a4470786e80a2a789e
Nov 03 23:25:16.469870 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:16.469837    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" event={"ID":"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c","Type":"ContainerStarted","Data":"3992ca097ce85ee07c2580a14d3090c791ab52aaabf956a4470786e80a2a789e"}
Nov 03 23:25:17.476663 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:17.476626    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" event={"ID":"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c","Type":"ContainerStarted","Data":"7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce"}
Nov 03 23:25:17.477048 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:17.476791    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:17.590940 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:17.590888    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" podStartSLOduration=2.582433607 podStartE2EDuration="3.590873801s" totalImagesPullingTime="1.008440194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:25:17.589376611 +0000 UTC m=+3720.854076621" watchObservedRunningTime="2026-11-03 23:25:17.590873801 +0000 UTC m=+3720.855573811"
Nov 03 23:25:19.921405 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:19.921350    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:23.486780 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:23.486752    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:25:24.828407 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:24.828353    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:29.627704 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:29.627619    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:32.555099 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:32.555069    5162 generic.go:356] "Generic (PLEG): container finished" podID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerID="b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1" exitCode=0
Nov 03 23:25:32.555584 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:32.555126    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" event={"ID":"0f801e00-641b-4b27-a7af-9faed4f1235f","Type":"ContainerDied","Data":"b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1"}
Nov 03 23:25:32.555584 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:32.555505    5162 scope.go:122] "RemoveContainer" containerID="b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1"
Nov 03 23:25:32.556587 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:32.556559    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:25:32.556704 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:32.556638    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:25:32.556704 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:32.556668    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:25:33.207944 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:33.207906    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:33.569997 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:33.569905    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" event={"ID":"0f801e00-641b-4b27-a7af-9faed4f1235f","Type":"ContainerStarted","Data":"82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177"}
Nov 03 23:25:33.764812 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:33.764755    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:33.915815 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:33.915726    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:33.917009 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:33.916980    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Nov 03 23:25:33.917184 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:33.917025    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Nov 03 23:25:34.019463 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:34.019423    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:34.812948 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:34.812913    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:34.915654 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:34.915618    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Nov 03 23:25:34.915847 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:34.915670    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Nov 03 23:25:35.916086 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:35.916053    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Nov 03 23:25:35.916528 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:35.916109    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Nov 03 23:25:36.916367 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:36.916337    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Nov 03 23:25:36.916863 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:36.916415    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Nov 03 23:25:37.092249 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:37.092194    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:37.420521 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:37.420483    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:37.916087 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:37.916014    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Nov 03 23:25:37.916087 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:37.916073    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Nov 03 23:25:38.750080 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:38.750047    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:38.920058 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:38.920032    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:38.921166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:38.921147    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:25:39.625878 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:39.625848    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:40.084074 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:25:40.084037    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:25:44.425187 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:44.425128    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:49.310982 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:49.310949    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:53.066533 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:53.066498    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:53.751914 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:53.751880    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:54.214401 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:54.214361    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:25:59.028273 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:25:59.028237    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:03.902283 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:03.902251    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:08.710448 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:08.710408    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:09.507174 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:09.507136    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:09.744776 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:09.744746    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:10.083329 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:26:10.083287    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:26:13.618494 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:13.618456    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:14.989681 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:14.989645    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:15.242101 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:15.242022    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:18.421623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:18.421577    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:23.317329 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:23.317290    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:27.787945 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:27.787907    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:28.129429 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:28.129301    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:32.813683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:32.813642    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:33.345917 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:33.345874    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:38.217985 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:38.217903    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:39.101116 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:39.101078    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:40.086525 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:26:40.086482    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:26:40.187955 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:40.187899    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:40.450456 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:40.450425    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:42.290995 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:42.290962    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"]
Nov 03 23:26:42.291443 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:42.291138    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" podUID="4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" containerName="maas-api" containerID="cri-o://7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce" gracePeriod=30
Nov 03 23:26:42.450868 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:42.450831    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:43.536707 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.536681    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:26:43.550113 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.550036    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-metrics-certs\") pod \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") "
Nov 03 23:26:43.550245 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.550219    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-kube-api-access-sf42n\" (UniqueName: \"kubernetes.io/projected/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-kube-api-access-sf42n\") pod \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") "
Nov 03 23:26:43.550339 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.550323    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls\") pod \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") "
Nov 03 23:26:43.550437 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.550362    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-openshift-service-ca\") pod \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\" (UID: \"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c\") "
Nov 03 23:26:43.550865 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.550834    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-openshift-service-ca" pod "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" (UID: "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:26:43.552098 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.552074    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-metrics-certs" pod "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" (UID: "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:26:43.552238 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.552216    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-kube-api-access-sf42n" pod "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" (UID: "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c"). InnerVolumeSpecName "kube-api-access-sf42n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:26:43.552295 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.552268    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls" pod "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" (UID: "4044f97c-7a5a-4b5e-a2dd-e17bfe91791c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:26:43.652224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.652139    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:43.652224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.652169    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:43.652224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.652182    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:43.652224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.652194    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf42n\" (UniqueName: \"kubernetes.io/projected/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c-kube-api-access-sf42n\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:43.924092 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.924062    5162 generic.go:356] "Generic (PLEG): container finished" podID="4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" containerID="7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce" exitCode=0
Nov 03 23:26:43.924270 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.924109    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"
Nov 03 23:26:43.924270 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.924143    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" event={"ID":"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c","Type":"ContainerDied","Data":"7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce"}
Nov 03 23:26:43.924270 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.924198    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" event={"ID":"4044f97c-7a5a-4b5e-a2dd-e17bfe91791c","Type":"ContainerDied","Data":"3992ca097ce85ee07c2580a14d3090c791ab52aaabf956a4470786e80a2a789e"}
Nov 03 23:26:43.924270 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.924221    5162 scope.go:122] "RemoveContainer" containerID="7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce"
Nov 03 23:26:43.933588 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.933566    5162 scope.go:122] "RemoveContainer" containerID="7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce"
Nov 03 23:26:43.933839 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:26:43.933820    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce\": container with ID starting with 7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce not found: ID does not exist" containerID="7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce"
Nov 03 23:26:43.933880 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.933848    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce"} err="failed to get container status \"7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce\": rpc error: code = NotFound desc = could not find container \"7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce\": container with ID starting with 7ff341ad537422c3f0473aadcf7aca630e3bf6d0200015fe93206228fc4dd6ce not found: ID does not exist"
Nov 03 23:26:43.985371 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:43.985346    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:44.085792 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:44.085763    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"]
Nov 03 23:26:44.118110 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:44.118078    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4"]
Nov 03 23:26:44.274275 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:44.274170    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:44.852739 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:44.852710    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" path="/var/lib/kubelet/pods/4044f97c-7a5a-4b5e-a2dd-e17bfe91791c/volumes"
Nov 03 23:26:45.022571 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:45.022534    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:46.483254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:46.483211    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-e433fd-d87dd6647-nvnz4" podUID="4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.116:8443/health\": context deadline exceeded"
Nov 03 23:26:51.059108 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:51.059071    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:51.307881 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:51.307853    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"]
Nov 03 23:26:51.308088 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:51.308062    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy" containerID="cri-o://82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177" gracePeriod=30
Nov 03 23:26:51.314354 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:51.314295    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:51.521219 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:51.521181    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:53.406330 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:53.406289    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:53.922565 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:53.922525    5162 patch_prober.go:29] interesting pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:26:53.922744 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:53.922589    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:26:56.557387 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.557363    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:26:56.670809 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.670780    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-socket\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.670988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.670832    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-kube-api-access-s9g7r\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-kube-api-access-s9g7r\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.670988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.670862    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-data\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.670988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.670889    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-envoy\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.670988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.670919    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-token\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-token\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.670988 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.670943    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-certs\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.671254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671067    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-credential-socket\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.671254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671067    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-socket" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:26:56.671254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671111    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-podinfo\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.671254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671170    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f801e00-641b-4b27-a7af-9faed4f1235f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f801e00-641b-4b27-a7af-9faed4f1235f-istiod-ca-cert\") pod \"0f801e00-641b-4b27-a7af-9faed4f1235f\" (UID: \"0f801e00-641b-4b27-a7af-9faed4f1235f\") "
Nov 03 23:26:56.671254 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671190    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-data" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:26:56.671565 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671325    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-credential-socket" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:26:56.671623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671571    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-certs" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:26:56.671623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671580    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.671623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671600    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f801e00-641b-4b27-a7af-9faed4f1235f-istiod-ca-cert" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:26:56.671739 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671624    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.671739 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.671643    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.673102 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.673082    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-token" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:26:56.673102 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.673086    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-envoy" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:26:56.673419 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.673375    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-kube-api-access-s9g7r" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "kube-api-access-s9g7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:26:56.673523 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.673498    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-podinfo" pod "0f801e00-641b-4b27-a7af-9faed4f1235f" (UID: "0f801e00-641b-4b27-a7af-9faed4f1235f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:26:56.772859 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.772827    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9g7r\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-kube-api-access-s9g7r\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.772859 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.772852    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.772859 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.772861    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.773083 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.772869    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f801e00-641b-4b27-a7af-9faed4f1235f-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.773083 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.772878    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f801e00-641b-4b27-a7af-9faed4f1235f-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.773083 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.772885    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f801e00-641b-4b27-a7af-9faed4f1235f-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:26:56.996176 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.996101    5162 generic.go:356] "Generic (PLEG): container finished" podID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerID="82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177" exitCode=0
Nov 03 23:26:56.996176 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.996147    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"
Nov 03 23:26:56.996176 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.996162    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" event={"ID":"0f801e00-641b-4b27-a7af-9faed4f1235f","Type":"ContainerDied","Data":"82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177"}
Nov 03 23:26:56.996404 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.996201    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l" event={"ID":"0f801e00-641b-4b27-a7af-9faed4f1235f","Type":"ContainerDied","Data":"7b03099a1fb74ea5f9e13c374a177dff16ba54a2aec8afd4c931688de7cd679b"}
Nov 03 23:26:56.996404 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:56.996222    5162 scope.go:122] "RemoveContainer" containerID="82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177"
Nov 03 23:26:57.005028 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.005011    5162 scope.go:122] "RemoveContainer" containerID="b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1"
Nov 03 23:26:57.015304 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.015270    5162 scope.go:122] "RemoveContainer" containerID="82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177"
Nov 03 23:26:57.015633 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:26:57.015613    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177\": container with ID starting with 82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177 not found: ID does not exist" containerID="82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177"
Nov 03 23:26:57.015703 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.015641    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177"} err="failed to get container status \"82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177\": rpc error: code = NotFound desc = could not find container \"82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177\": container with ID starting with 82b34276cb32605f25131f165103482dba245fb8696f579aa71c7a034bf48177 not found: ID does not exist"
Nov 03 23:26:57.015703 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.015665    5162 scope.go:122] "RemoveContainer" containerID="b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1"
Nov 03 23:26:57.015946 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:26:57.015924    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1\": container with ID starting with b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1 not found: ID does not exist" containerID="b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1"
Nov 03 23:26:57.015986 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.015956    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1"} err="failed to get container status \"b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1\": rpc error: code = NotFound desc = could not find container \"b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1\": container with ID starting with b52955d4a0b9d3a5338c1db1cdc2324cbc91bfd041e739d991a19c0d30420ff1 not found: ID does not exist"
Nov 03 23:26:57.143356 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143327    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"]
Nov 03 23:26:57.143788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143771    5162 cpu_manager.go:414] "containerMap: removing container" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy"
Nov 03 23:26:57.143918 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143791    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy"
Nov 03 23:26:57.143918 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143827    5162 cpu_manager.go:414] "containerMap: removing container" podUID="4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" containerName="maas-api"
Nov 03 23:26:57.143918 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143835    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" containerName="maas-api"
Nov 03 23:26:57.144117 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143963    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy"
Nov 03 23:26:57.144117 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143981    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy"
Nov 03 23:26:57.144117 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.143991    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="4044f97c-7a5a-4b5e-a2dd-e17bfe91791c" containerName="maas-api"
Nov 03 23:26:57.148619 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.148601    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.154994 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.154975    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"]
Nov 03 23:26:57.190981 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.190960    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l"]
Nov 03 23:26:57.200355 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.200335    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d1cbe6a2-kserve-self-signed-certs\""
Nov 03 23:26:57.202702 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.202682    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"]
Nov 03 23:26:57.232208 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.232176    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:26:57.278454 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.278371    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78vks\" (UniqueName: \"kubernetes.io/projected/3a86bf65-4209-496e-9618-39637d71e5a1-kube-api-access-78vks\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.278600 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.278466    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-dshm\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.278600 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.278510    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a86bf65-4209-496e-9618-39637d71e5a1-tls-certs\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.278600 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.278576    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-tmp-dir\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.278700 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.278616    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-model-cache\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.278700 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.278657    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-home\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.379831 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.379800    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78vks\" (UniqueName: \"kubernetes.io/projected/3a86bf65-4209-496e-9618-39637d71e5a1-kube-api-access-78vks\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.379831 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.379836    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-dshm\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.380055 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.379863    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a86bf65-4209-496e-9618-39637d71e5a1-tls-certs\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.380055 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.379897    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-tmp-dir\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.380055 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.379938    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-model-cache\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.380055 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.379988    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-home\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.380322 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.380294    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-tmp-dir\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.380446 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.380321    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-home\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.380446 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.380370    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-model-cache\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.382181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.382161    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-dshm\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.382315 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.382299    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a86bf65-4209-496e-9618-39637d71e5a1-tls-certs\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.438309 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.438276    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78vks\" (UniqueName: \"kubernetes.io/projected/3a86bf65-4209-496e-9618-39637d71e5a1-kube-api-access-78vks\") pod \"e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") " pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.461437 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.461385    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:57.837944 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:57.837915    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"]
Nov 03 23:26:57.838633 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:26:57.838609    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a86bf65_4209_496e_9618_39637d71e5a1.slice/crio-3c01adca9df3c6c6102a29d5f1565e105f6a12825ca9db4c077408c6c23e5816 WatchSource:0}: Error finding container 3c01adca9df3c6c6102a29d5f1565e105f6a12825ca9db4c077408c6c23e5816: Status 404 returned error can't find the container with id 3c01adca9df3c6c6102a29d5f1565e105f6a12825ca9db4c077408c6c23e5816
Nov 03 23:26:58.004348 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:58.004309    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl" event={"ID":"3a86bf65-4209-496e-9618-39637d71e5a1","Type":"ContainerStarted","Data":"b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86"}
Nov 03 23:26:58.004348 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:58.004354    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl" event={"ID":"3a86bf65-4209-496e-9618-39637d71e5a1","Type":"ContainerStarted","Data":"3c01adca9df3c6c6102a29d5f1565e105f6a12825ca9db4c077408c6c23e5816"}
Nov 03 23:26:58.120030 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:58.119939    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl" podStartSLOduration=1.119926535 podStartE2EDuration="1.119926535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:26:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:26:58.118428723 +0000 UTC m=+3821.383128732" watchObservedRunningTime="2026-11-03 23:26:58.119926535 +0000 UTC m=+3821.384626544"
Nov 03 23:26:58.462192 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:58.462151    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:26:58.852909 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:26:58.852833    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" path="/var/lib/kubelet/pods/0f801e00-641b-4b27-a7af-9faed4f1235f/volumes"
Nov 03 23:27:07.462451 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:07.462409    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:27:07.474763 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:07.474735    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:27:08.069283 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:08.069252    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:27:09.702342 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:09.702309    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:27:10.038155 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:10.038065    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:27:10.084213 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:27:10.084179    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:27:24.262628 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:24.262585    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:27:39.460059 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:39.460023    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:27:40.083899 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:27:40.083861    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:27:40.321106 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.321076    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"]
Nov 03 23:27:40.321512 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.321497    5162 cpu_manager.go:414] "containerMap: removing container" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy"
Nov 03 23:27:40.321573 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.321513    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f801e00-641b-4b27-a7af-9faed4f1235f" containerName="istio-proxy"
Nov 03 23:27:40.324543 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.324518    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.424963 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.424936    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d1cbe6a2-kserve-self-signed-certs\""
Nov 03 23:27:40.425496 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.425473    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-dshm\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.425611 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.425528    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-tmp-dir\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.425669 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.425634    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crxm5\" (UniqueName: \"kubernetes.io/projected/a389b6d7-f472-4c87-88e5-3b77293df051-kube-api-access-crxm5\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.425725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.425671    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-model-cache\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.425725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.425705    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-home\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.425798 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.425791    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a389b6d7-f472-4c87-88e5-3b77293df051-tls-certs\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.427419 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.427382    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"]
Nov 03 23:27:40.472944 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.472907    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:27:40.526382 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.526352    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a389b6d7-f472-4c87-88e5-3b77293df051-tls-certs\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.526607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.526457    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-dshm\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.526607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.526504    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-tmp-dir\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.526607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.526570    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crxm5\" (UniqueName: \"kubernetes.io/projected/a389b6d7-f472-4c87-88e5-3b77293df051-kube-api-access-crxm5\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.526607 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.526601    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-model-cache\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.526827 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.526636    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-home\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.526981 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.526954    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-tmp-dir\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.527066 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.527019    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-model-cache\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.527126 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.527068    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-home\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.528813 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.528785    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-dshm\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.528952 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.528936    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a389b6d7-f472-4c87-88e5-3b77293df051-tls-certs\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.589309 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.589282    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crxm5\" (UniqueName: \"kubernetes.io/projected/a389b6d7-f472-4c87-88e5-3b77293df051-kube-api-access-crxm5\") pod \"e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") " pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:40.635883 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:40.635861    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:41.012482 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:41.012455    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"]
Nov 03 23:27:41.013652 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:27:41.013623    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda389b6d7_f472_4c87_88e5_3b77293df051.slice/crio-9f8f91234cff87ab4aec131c6055c28f9b88814c582a0e815d325c28b0729200 WatchSource:0}: Error finding container 9f8f91234cff87ab4aec131c6055c28f9b88814c582a0e815d325c28b0729200: Status 404 returned error can't find the container with id 9f8f91234cff87ab4aec131c6055c28f9b88814c582a0e815d325c28b0729200
Nov 03 23:27:41.230825 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:41.230793    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq" event={"ID":"a389b6d7-f472-4c87-88e5-3b77293df051","Type":"ContainerStarted","Data":"8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f"}
Nov 03 23:27:41.230992 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:41.230836    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq" event={"ID":"a389b6d7-f472-4c87-88e5-3b77293df051","Type":"ContainerStarted","Data":"9f8f91234cff87ab4aec131c6055c28f9b88814c582a0e815d325c28b0729200"}
Nov 03 23:27:41.343446 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:41.343330    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq" podStartSLOduration=1.343317478 podStartE2EDuration="1.343317478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:27:41.342039028 +0000 UTC m=+3864.606739039" watchObservedRunningTime="2026-11-03 23:27:41.343317478 +0000 UTC m=+3864.608017489"
Nov 03 23:27:41.636159 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:41.636077    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:50.635970 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:50.635931    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:50.649902 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:50.649881    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:51.337260 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:51.337229    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:27:53.694852 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:53.694814    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:27:59.103072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:27:59.103033    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:28:10.083231 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:28:10.083143    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:28:37.256527 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:37.256497    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"]
Nov 03 23:28:37.351248 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:37.346802    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:28:39.807550 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:39.807512    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:28:40.084938 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:28:40.084856    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:28:40.847922 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:40.847885    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:28:52.276179 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.276118    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl" podUID="3a86bf65-4209-496e-9618-39637d71e5a1" containerName="main" containerID="cri-o://b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86" gracePeriod=45
Nov 03 23:28:52.530469 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.530419    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:28:52.591222 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591194    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-dshm\") pod \"3a86bf65-4209-496e-9618-39637d71e5a1\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") "
Nov 03 23:28:52.591375 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591233    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a86bf65-4209-496e-9618-39637d71e5a1-kube-api-access-78vks\" (UniqueName: \"kubernetes.io/projected/3a86bf65-4209-496e-9618-39637d71e5a1-kube-api-access-78vks\") pod \"3a86bf65-4209-496e-9618-39637d71e5a1\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") "
Nov 03 23:28:52.591375 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591354    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-tmp-dir\") pod \"3a86bf65-4209-496e-9618-39637d71e5a1\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") "
Nov 03 23:28:52.591495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591458    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-home\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-home\") pod \"3a86bf65-4209-496e-9618-39637d71e5a1\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") "
Nov 03 23:28:52.591540 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591511    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-model-cache\") pod \"3a86bf65-4209-496e-9618-39637d71e5a1\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") "
Nov 03 23:28:52.591594 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591552    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a86bf65-4209-496e-9618-39637d71e5a1-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a86bf65-4209-496e-9618-39637d71e5a1-tls-certs\") pod \"3a86bf65-4209-496e-9618-39637d71e5a1\" (UID: \"3a86bf65-4209-496e-9618-39637d71e5a1\") "
Nov 03 23:28:52.591762 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591658    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-tmp-dir" pod "3a86bf65-4209-496e-9618-39637d71e5a1" (UID: "3a86bf65-4209-496e-9618-39637d71e5a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:28:52.591762 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591731    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-home" pod "3a86bf65-4209-496e-9618-39637d71e5a1" (UID: "3a86bf65-4209-496e-9618-39637d71e5a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:28:52.591935 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.591823    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-model-cache" pod "3a86bf65-4209-496e-9618-39637d71e5a1" (UID: "3a86bf65-4209-496e-9618-39637d71e5a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:28:52.592029 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.592003    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:28:52.592029 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.592029    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:28:52.592208 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.592042    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:28:52.593622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.593597    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-dshm" pod "3a86bf65-4209-496e-9618-39637d71e5a1" (UID: "3a86bf65-4209-496e-9618-39637d71e5a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:28:52.593720 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.593702    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a86bf65-4209-496e-9618-39637d71e5a1-kube-api-access-78vks" pod "3a86bf65-4209-496e-9618-39637d71e5a1" (UID: "3a86bf65-4209-496e-9618-39637d71e5a1"). InnerVolumeSpecName "kube-api-access-78vks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:28:52.593766 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.593737    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a86bf65-4209-496e-9618-39637d71e5a1-tls-certs" pod "3a86bf65-4209-496e-9618-39637d71e5a1" (UID: "3a86bf65-4209-496e-9618-39637d71e5a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:28:52.693181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.693145    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a86bf65-4209-496e-9618-39637d71e5a1-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:28:52.693181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.693174    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a86bf65-4209-496e-9618-39637d71e5a1-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:28:52.693181 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.693188    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78vks\" (UniqueName: \"kubernetes.io/projected/3a86bf65-4209-496e-9618-39637d71e5a1-kube-api-access-78vks\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:28:52.716534 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.716498    5162 generic.go:356] "Generic (PLEG): container finished" podID="3a86bf65-4209-496e-9618-39637d71e5a1" containerID="b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86" exitCode=0
Nov 03 23:28:52.716677 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.716544    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"
Nov 03 23:28:52.716677 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.716579    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl" event={"ID":"3a86bf65-4209-496e-9618-39637d71e5a1","Type":"ContainerDied","Data":"b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86"}
Nov 03 23:28:52.716677 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.716621    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl" event={"ID":"3a86bf65-4209-496e-9618-39637d71e5a1","Type":"ContainerDied","Data":"3c01adca9df3c6c6102a29d5f1565e105f6a12825ca9db4c077408c6c23e5816"}
Nov 03 23:28:52.716677 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.716638    5162 scope.go:122] "RemoveContainer" containerID="b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86"
Nov 03 23:28:52.726022 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.726006    5162 scope.go:122] "RemoveContainer" containerID="b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86"
Nov 03 23:28:52.726318 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:28:52.726299    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86\": container with ID starting with b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86 not found: ID does not exist" containerID="b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86"
Nov 03 23:28:52.726378 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.726326    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86"} err="failed to get container status \"b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86\": rpc error: code = NotFound desc = could not find container \"b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86\": container with ID starting with b2b0af0eee62b5e5224907be1719b9aac6c4bacd3885029da1bd371b435e8b86 not found: ID does not exist"
Nov 03 23:28:52.874329 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.874251    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"]
Nov 03 23:28:52.878632 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:52.878600    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d1cbe6a2-kserve-745cdd854f-ksftl"]
Nov 03 23:28:54.115690 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:54.115651    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:28:54.852946 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:54.852915    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a86bf65-4209-496e-9618-39637d71e5a1" path="/var/lib/kubelet/pods/3a86bf65-4209-496e-9618-39637d71e5a1/volumes"
Nov 03 23:28:55.485865 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:55.485830    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"]
Nov 03 23:28:55.585174 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:55.585133    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:28:59.449565 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:28:59.449526    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:10.083627 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:29:10.083586    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:29:10.505298 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.505257    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq" podUID="a389b6d7-f472-4c87-88e5-3b77293df051" containerName="main" containerID="cri-o://8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f" gracePeriod=45
Nov 03 23:29:10.755759 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.755703    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:29:10.783287 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783262    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-dshm\") pod \"a389b6d7-f472-4c87-88e5-3b77293df051\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") "
Nov 03 23:29:10.783436 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783314    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-model-cache\") pod \"a389b6d7-f472-4c87-88e5-3b77293df051\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") "
Nov 03 23:29:10.783515 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783449    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-tmp-dir\") pod \"a389b6d7-f472-4c87-88e5-3b77293df051\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") "
Nov 03 23:29:10.783577 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783541    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a389b6d7-f472-4c87-88e5-3b77293df051-tls-certs\" (UniqueName: \"kubernetes.io/secret/a389b6d7-f472-4c87-88e5-3b77293df051-tls-certs\") pod \"a389b6d7-f472-4c87-88e5-3b77293df051\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") "
Nov 03 23:29:10.783577 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783561    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-home\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-home\") pod \"a389b6d7-f472-4c87-88e5-3b77293df051\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") "
Nov 03 23:29:10.783697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783582    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-model-cache" pod "a389b6d7-f472-4c87-88e5-3b77293df051" (UID: "a389b6d7-f472-4c87-88e5-3b77293df051"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:29:10.783697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783606    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a389b6d7-f472-4c87-88e5-3b77293df051-kube-api-access-crxm5\" (UniqueName: \"kubernetes.io/projected/a389b6d7-f472-4c87-88e5-3b77293df051-kube-api-access-crxm5\") pod \"a389b6d7-f472-4c87-88e5-3b77293df051\" (UID: \"a389b6d7-f472-4c87-88e5-3b77293df051\") "
Nov 03 23:29:10.783697 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783670    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-tmp-dir" pod "a389b6d7-f472-4c87-88e5-3b77293df051" (UID: "a389b6d7-f472-4c87-88e5-3b77293df051"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:29:10.783865 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783815    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-home" pod "a389b6d7-f472-4c87-88e5-3b77293df051" (UID: "a389b6d7-f472-4c87-88e5-3b77293df051"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:29:10.783989 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783973    5162 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-model-cache\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:29:10.784050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.783993    5162 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-tmp-dir\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:29:10.784050 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.784007    5162 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-home\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:29:10.785431 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.785407    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-dshm" pod "a389b6d7-f472-4c87-88e5-3b77293df051" (UID: "a389b6d7-f472-4c87-88e5-3b77293df051"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:29:10.785516 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.785443    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a389b6d7-f472-4c87-88e5-3b77293df051-tls-certs" pod "a389b6d7-f472-4c87-88e5-3b77293df051" (UID: "a389b6d7-f472-4c87-88e5-3b77293df051"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:29:10.785659 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.785640    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a389b6d7-f472-4c87-88e5-3b77293df051-kube-api-access-crxm5" pod "a389b6d7-f472-4c87-88e5-3b77293df051" (UID: "a389b6d7-f472-4c87-88e5-3b77293df051"). InnerVolumeSpecName "kube-api-access-crxm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:29:10.829137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.829111    5162 generic.go:356] "Generic (PLEG): container finished" podID="a389b6d7-f472-4c87-88e5-3b77293df051" containerID="8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f" exitCode=0
Nov 03 23:29:10.829264 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.829154    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"
Nov 03 23:29:10.829264 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.829199    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq" event={"ID":"a389b6d7-f472-4c87-88e5-3b77293df051","Type":"ContainerDied","Data":"8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f"}
Nov 03 23:29:10.829264 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.829240    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq" event={"ID":"a389b6d7-f472-4c87-88e5-3b77293df051","Type":"ContainerDied","Data":"9f8f91234cff87ab4aec131c6055c28f9b88814c582a0e815d325c28b0729200"}
Nov 03 23:29:10.829264 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.829261    5162 scope.go:122] "RemoveContainer" containerID="8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f"
Nov 03 23:29:10.839083 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.839066    5162 scope.go:122] "RemoveContainer" containerID="8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f"
Nov 03 23:29:10.839341 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:29:10.839320    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f\": container with ID starting with 8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f not found: ID does not exist" containerID="8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f"
Nov 03 23:29:10.839440 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.839353    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f"} err="failed to get container status \"8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f\": rpc error: code = NotFound desc = could not find container \"8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f\": container with ID starting with 8c2b4c8f329ebe0035c6b36a313c3849d838cacd70275c779efbe9fbe9fe8d6f not found: ID does not exist"
Nov 03 23:29:10.885023 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.884994    5162 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a389b6d7-f472-4c87-88e5-3b77293df051-dshm\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:29:10.885023 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.885017    5162 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a389b6d7-f472-4c87-88e5-3b77293df051-tls-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:29:10.885253 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.885032    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crxm5\" (UniqueName: \"kubernetes.io/projected/a389b6d7-f472-4c87-88e5-3b77293df051-kube-api-access-crxm5\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:29:10.974207 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.974181    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"]
Nov 03 23:29:10.980461 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:10.980430    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d1cbe6a2-kserve-56585c9bf-v8vkq"]
Nov 03 23:29:12.853885 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:12.853853    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a389b6d7-f472-4c87-88e5-3b77293df051" path="/var/lib/kubelet/pods/a389b6d7-f472-4c87-88e5-3b77293df051/volumes"
Nov 03 23:29:40.082506 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:29:40.082428    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:29:40.194843 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:40.194809    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:45.344338 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.344307    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"]
Nov 03 23:29:45.344760 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.344732    5162 cpu_manager.go:414] "containerMap: removing container" podUID="3a86bf65-4209-496e-9618-39637d71e5a1" containerName="main"
Nov 03 23:29:45.344760 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.344744    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a86bf65-4209-496e-9618-39637d71e5a1" containerName="main"
Nov 03 23:29:45.344760 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.344755    5162 cpu_manager.go:414] "containerMap: removing container" podUID="a389b6d7-f472-4c87-88e5-3b77293df051" containerName="main"
Nov 03 23:29:45.344760 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.344760    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a389b6d7-f472-4c87-88e5-3b77293df051" containerName="main"
Nov 03 23:29:45.344896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.344825    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a86bf65-4209-496e-9618-39637d71e5a1" containerName="main"
Nov 03 23:29:45.344896 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.344834    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="a389b6d7-f472-4c87-88e5-3b77293df051" containerName="main"
Nov 03 23:29:45.347937 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.347921    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.445066 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.445034    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98136380-67c7-4701-8cea-6b44a8ff4802-openshift-service-ca\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.445226 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.445078    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98136380-67c7-4701-8cea-6b44a8ff4802-metrics-certs\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.445226 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.445170    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98136380-67c7-4701-8cea-6b44a8ff4802-maas-api-tls\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.445381 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.445360    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwvcv\" (UniqueName: \"kubernetes.io/projected/98136380-67c7-4701-8cea-6b44a8ff4802-kube-api-access-pwvcv\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.447725 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.447706    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"]
Nov 03 23:29:45.546160 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.546119    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98136380-67c7-4701-8cea-6b44a8ff4802-openshift-service-ca\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.546352 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.546167    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98136380-67c7-4701-8cea-6b44a8ff4802-metrics-certs\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.546352 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.546197    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98136380-67c7-4701-8cea-6b44a8ff4802-maas-api-tls\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.546352 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.546332    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvcv\" (UniqueName: \"kubernetes.io/projected/98136380-67c7-4701-8cea-6b44a8ff4802-kube-api-access-pwvcv\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.546926 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.546902    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98136380-67c7-4701-8cea-6b44a8ff4802-openshift-service-ca\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.548652 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.548623    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98136380-67c7-4701-8cea-6b44a8ff4802-maas-api-tls\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.548794 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.548771    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98136380-67c7-4701-8cea-6b44a8ff4802-metrics-certs\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.605123 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.605062    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwvcv\" (UniqueName: \"kubernetes.io/projected/98136380-67c7-4701-8cea-6b44a8ff4802-kube-api-access-pwvcv\") pod \"maas-api-5b4b4b5f89-pz627\" (UID: \"98136380-67c7-4701-8cea-6b44a8ff4802\") " pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.660017 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.659990    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:45.746291 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:45.746249    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:46.030501 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:46.030471    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"]
Nov 03 23:29:46.031983 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:29:46.031957    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98136380_67c7_4701_8cea_6b44a8ff4802.slice/crio-368e2a5dd0f311386280bd9e2657e857447c2acd84fa82427823d818d68bea54 WatchSource:0}: Error finding container 368e2a5dd0f311386280bd9e2657e857447c2acd84fa82427823d818d68bea54: Status 404 returned error can't find the container with id 368e2a5dd0f311386280bd9e2657e857447c2acd84fa82427823d818d68bea54
Nov 03 23:29:46.033255 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:46.033238    5162 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 23:29:46.054589 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:46.054560    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627" event={"ID":"98136380-67c7-4701-8cea-6b44a8ff4802","Type":"ContainerStarted","Data":"368e2a5dd0f311386280bd9e2657e857447c2acd84fa82427823d818d68bea54"}
Nov 03 23:29:47.061486 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:47.061451    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627" event={"ID":"98136380-67c7-4701-8cea-6b44a8ff4802","Type":"ContainerStarted","Data":"f0fe6d5fdf3f8249d3031e0412fc57ac5b84115365f8075252601a5877d9fb07"}
Nov 03 23:29:47.061889 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:47.061610    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:47.179941 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:47.179890    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627" podStartSLOduration=1.267756222 podStartE2EDuration="2.179878309s" totalImagesPullingTime="912.122087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:29:47.178481145 +0000 UTC m=+3990.443181155" watchObservedRunningTime="2026-11-03 23:29:47.179878309 +0000 UTC m=+3990.444578319"
Nov 03 23:29:51.914776 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:51.914737    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:52.492316 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:52.492266    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:52.742943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:52.742854    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:53.072073 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:53.072006    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b4b4b5f89-pz627"
Nov 03 23:29:54.447916 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:54.447882    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:56.408188 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:56.408149    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:56.615053 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:56.615022    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:59.632713 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:59.632680    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:29:59.884293 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:29:59.884216    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:00.134156 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.134115    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"]
Nov 03 23:30:00.138218 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.138158    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.189949 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.189925    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-cc1c5661\""
Nov 03 23:30:00.192662 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.192632    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4wj6\" (UniqueName: \"kubernetes.io/projected/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-kube-api-access-x4wj6\") pod \"payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") " pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.192811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.192702    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-plugins-config-volume\") pod \"payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") " pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.237798 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.237772    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-cc1c5661-dockercfg-kjzwp\""
Nov 03 23:30:00.240074 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.240052    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"]
Nov 03 23:30:00.290504 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.290476    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"]
Nov 03 23:30:00.293996 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.293975    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4wj6\" (UniqueName: \"kubernetes.io/projected/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-kube-api-access-x4wj6\") pod \"payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") " pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.294099 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.294015    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-plugins-config-volume\") pod \"payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") " pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.294205 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.294189    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.294784 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.294760    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-plugins-config-volume\") pod \"payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") " pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.352310 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.352280    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"]
Nov 03 23:30:00.367928 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.367902    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4wj6\" (UniqueName: \"kubernetes.io/projected/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-kube-api-access-x4wj6\") pod \"payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") " pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.390196 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.390131    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-cc1c5661-serving-cert\""
Nov 03 23:30:00.395486 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.395462    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-metrics-certs\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.395591 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.395509    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.395591 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.395527    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e38def-1ad5-4872-ad8c-d231cb8da850-openshift-service-ca\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.395754 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.395735    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6fqh\" (UniqueName: \"kubernetes.io/projected/93e38def-1ad5-4872-ad8c-d231cb8da850-kube-api-access-q6fqh\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.450985 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.450945    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:00.496858 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.496832    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6fqh\" (UniqueName: \"kubernetes.io/projected/93e38def-1ad5-4872-ad8c-d231cb8da850-kube-api-access-q6fqh\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.497013 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.496905    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-metrics-certs\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.497013 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.496941    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.497013 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.496958    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e38def-1ad5-4872-ad8c-d231cb8da850-openshift-service-ca\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.497226 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:30:00.497087    5162 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-serving-cert: secret "maas-api-e2e-ait-cc1c5661-serving-cert" not found
Nov 03 23:30:00.497226 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:30:00.497148    5162 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls podName:93e38def-1ad5-4872-ad8c-d231cb8da850 nodeName:}" failed. No retries permitted until 2026-11-03 23:30:00.997130509 +0000 UTC m=+4004.261830511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls") pod "maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" (UID: "93e38def-1ad5-4872-ad8c-d231cb8da850") : secret "maas-api-e2e-ait-cc1c5661-serving-cert" not found
Nov 03 23:30:00.497662 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.497642    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e38def-1ad5-4872-ad8c-d231cb8da850-openshift-service-ca\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.499256 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.499231    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-metrics-certs\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.546414 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.546299    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:00.560152 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.560105    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6fqh\" (UniqueName: \"kubernetes.io/projected/93e38def-1ad5-4872-ad8c-d231cb8da850-kube-api-access-q6fqh\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:00.752592 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.752540    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:00.824842 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.824819    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"]
Nov 03 23:30:00.826292 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:30:00.826263    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeac2fcd_aaad_45eb_846b_d42c8d5ef24b.slice/crio-05d82cbec6759073e457b5f3418089cddab63e6669f7f4d37ace42952df8d06a WatchSource:0}: Error finding container 05d82cbec6759073e457b5f3418089cddab63e6669f7f4d37ace42952df8d06a: Status 404 returned error can't find the container with id 05d82cbec6759073e457b5f3418089cddab63e6669f7f4d37ace42952df8d06a
Nov 03 23:30:00.956580 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:00.956547    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:01.002721 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.002669    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:01.004815 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.004795    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls\") pod \"maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:01.137313 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.137283    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv" event={"ID":"beac2fcd-aaad-45eb-846b-d42c8d5ef24b","Type":"ContainerStarted","Data":"6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d"}
Nov 03 23:30:01.137536 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.137333    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv" event={"ID":"beac2fcd-aaad-45eb-846b-d42c8d5ef24b","Type":"ContainerStarted","Data":"05d82cbec6759073e457b5f3418089cddab63e6669f7f4d37ace42952df8d06a"}
Nov 03 23:30:01.137536 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.137430    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:01.207288 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.207265    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:01.253277 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.253142    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv" podStartSLOduration=1.253121584 podStartE2EDuration="1.253121584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:30:01.250409733 +0000 UTC m=+4004.515109749" watchObservedRunningTime="2026-11-03 23:30:01.253121584 +0000 UTC m=+4004.517821596"
Nov 03 23:30:01.576778 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.576749    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"]
Nov 03 23:30:01.577602 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:30:01.577579    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e38def_1ad5_4872_ad8c_d231cb8da850.slice/crio-9fa58a7b0449acb26e0cf018037f26b5253ec6622537f79ac6a1f965dc00da06 WatchSource:0}: Error finding container 9fa58a7b0449acb26e0cf018037f26b5253ec6622537f79ac6a1f965dc00da06: Status 404 returned error can't find the container with id 9fa58a7b0449acb26e0cf018037f26b5253ec6622537f79ac6a1f965dc00da06
Nov 03 23:30:01.852246 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.852171    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"]
Nov 03 23:30:01.852659 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:01.852342    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798" podUID="44df9647-34b3-425f-b3f8-82719b9b14e0" containerName="maas-api" containerID="cri-o://3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152" gracePeriod=30
Nov 03 23:30:02.143851 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:02.143770    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" event={"ID":"93e38def-1ad5-4872-ad8c-d231cb8da850","Type":"ContainerStarted","Data":"9fa58a7b0449acb26e0cf018037f26b5253ec6622537f79ac6a1f965dc00da06"}
Nov 03 23:30:03.104441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.104413    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 23:30:03.124741 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.124718    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44df9647-34b3-425f-b3f8-82719b9b14e0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44df9647-34b3-425f-b3f8-82719b9b14e0-openshift-service-ca\") pod \"44df9647-34b3-425f-b3f8-82719b9b14e0\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") "
Nov 03 23:30:03.124853 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.124841    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-metrics-certs\") pod \"44df9647-34b3-425f-b3f8-82719b9b14e0\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") "
Nov 03 23:30:03.124920 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.124908    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-maas-api-tls\") pod \"44df9647-34b3-425f-b3f8-82719b9b14e0\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") "
Nov 03 23:30:03.124986 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.124930    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44df9647-34b3-425f-b3f8-82719b9b14e0-kube-api-access-879kj\" (UniqueName: \"kubernetes.io/projected/44df9647-34b3-425f-b3f8-82719b9b14e0-kube-api-access-879kj\") pod \"44df9647-34b3-425f-b3f8-82719b9b14e0\" (UID: \"44df9647-34b3-425f-b3f8-82719b9b14e0\") "
Nov 03 23:30:03.125113 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.125086    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44df9647-34b3-425f-b3f8-82719b9b14e0-openshift-service-ca" pod "44df9647-34b3-425f-b3f8-82719b9b14e0" (UID: "44df9647-34b3-425f-b3f8-82719b9b14e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:30:03.125363 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.125338    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44df9647-34b3-425f-b3f8-82719b9b14e0-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:03.127000 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.126978    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-maas-api-tls" pod "44df9647-34b3-425f-b3f8-82719b9b14e0" (UID: "44df9647-34b3-425f-b3f8-82719b9b14e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:30:03.127094 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.127060    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-metrics-certs" pod "44df9647-34b3-425f-b3f8-82719b9b14e0" (UID: "44df9647-34b3-425f-b3f8-82719b9b14e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:30:03.127094 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.127075    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44df9647-34b3-425f-b3f8-82719b9b14e0-kube-api-access-879kj" pod "44df9647-34b3-425f-b3f8-82719b9b14e0" (UID: "44df9647-34b3-425f-b3f8-82719b9b14e0"). InnerVolumeSpecName "kube-api-access-879kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:30:03.149654 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.149628    5162 generic.go:356] "Generic (PLEG): container finished" podID="44df9647-34b3-425f-b3f8-82719b9b14e0" containerID="3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152" exitCode=0
Nov 03 23:30:03.149757 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.149676    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"
Nov 03 23:30:03.149757 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.149704    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798" event={"ID":"44df9647-34b3-425f-b3f8-82719b9b14e0","Type":"ContainerDied","Data":"3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152"}
Nov 03 23:30:03.149757 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.149752    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-797d89b4d5-mx798" event={"ID":"44df9647-34b3-425f-b3f8-82719b9b14e0","Type":"ContainerDied","Data":"62c0047bfff97f1023e34c1f7ef568dc074cf29c585d7187b0ef2d6e580f940d"}
Nov 03 23:30:03.149898 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.149779    5162 scope.go:122] "RemoveContainer" containerID="3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152"
Nov 03 23:30:03.151463 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.151432    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" event={"ID":"93e38def-1ad5-4872-ad8c-d231cb8da850","Type":"ContainerStarted","Data":"1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c"}
Nov 03 23:30:03.151564 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.151546    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:03.160010 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.159991    5162 scope.go:122] "RemoveContainer" containerID="3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152"
Nov 03 23:30:03.160289 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:30:03.160269    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152\": container with ID starting with 3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152 not found: ID does not exist" containerID="3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152"
Nov 03 23:30:03.160370 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.160297    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152"} err="failed to get container status \"3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152\": rpc error: code = NotFound desc = could not find container \"3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152\": container with ID starting with 3770866a0d9d10b629a65b345e1ba71e3575b3bf432686beab23ddf7a59d1152 not found: ID does not exist"
Nov 03 23:30:03.226723 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.226688    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:03.226723 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.226716    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-879kj\" (UniqueName: \"kubernetes.io/projected/44df9647-34b3-425f-b3f8-82719b9b14e0-kube-api-access-879kj\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:03.226723 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.226726    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44df9647-34b3-425f-b3f8-82719b9b14e0-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:03.327663 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.327631    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"]
Nov 03 23:30:03.333013 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.332976    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-797d89b4d5-mx798"]
Nov 03 23:30:03.443166 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:03.443112    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" podStartSLOduration=2.53764506 podStartE2EDuration="3.443098344s" totalImagesPullingTime="905.453284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:30:03.441571877 +0000 UTC m=+4006.706271888" watchObservedRunningTime="2026-11-03 23:30:03.443098344 +0000 UTC m=+4006.707798355"
Nov 03 23:30:04.853603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:04.853569    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44df9647-34b3-425f-b3f8-82719b9b14e0" path="/var/lib/kubelet/pods/44df9647-34b3-425f-b3f8-82719b9b14e0/volumes"
Nov 03 23:30:05.916238 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:05.916202    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:09.162622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:09.162595    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:09.818806 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:09.818768    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:10.082744 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:30:10.082656    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:30:10.817626 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:10.817590    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:12.146778 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:12.146749    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:15.611952 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:15.611921    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:20.718075 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:20.718039    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:25.717382 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:25.717349    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:30.528523 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:30.528476    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:35.353031 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:35.352998    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:40.083485 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:30:40.083442    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:30:40.118402 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:40.118353    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:44.036046 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:44.036013    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"]
Nov 03 23:30:44.502984 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:44.502948    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:44.745318 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:44.745280    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:44.963268 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:44.963240    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:49.036371 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.036328    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv" podUID="beac2fcd-aaad-45eb-846b-d42c8d5ef24b" containerName="payload-processing" containerID="cri-o://6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d" gracePeriod=25
Nov 03 23:30:49.036854 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.036752    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"]
Nov 03 23:30:49.037191 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.036993    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" podUID="93e38def-1ad5-4872-ad8c-d231cb8da850" containerName="maas-api" containerID="cri-o://1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c" gracePeriod=30
Nov 03 23:30:49.157878 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.157844    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" podUID="93e38def-1ad5-4872-ad8c-d231cb8da850" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.121:8443/health\": dial tcp 10.134.0.121:8443: connect: connection refused"
Nov 03 23:30:49.196603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.196567    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:49.317500 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.317477    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:49.320734 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.320715    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:49.376651 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.376626    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-plugins-config-volume\") pod \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") "
Nov 03 23:30:49.376844 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.376674    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93e38def-1ad5-4872-ad8c-d231cb8da850-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e38def-1ad5-4872-ad8c-d231cb8da850-openshift-service-ca\") pod \"93e38def-1ad5-4872-ad8c-d231cb8da850\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") "
Nov 03 23:30:49.376844 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.376698    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-kube-api-access-x4wj6\" (UniqueName: \"kubernetes.io/projected/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-kube-api-access-x4wj6\") pod \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\" (UID: \"beac2fcd-aaad-45eb-846b-d42c8d5ef24b\") "
Nov 03 23:30:49.376844 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.376735    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls\") pod \"93e38def-1ad5-4872-ad8c-d231cb8da850\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") "
Nov 03 23:30:49.376844 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.376766    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93e38def-1ad5-4872-ad8c-d231cb8da850-kube-api-access-q6fqh\" (UniqueName: \"kubernetes.io/projected/93e38def-1ad5-4872-ad8c-d231cb8da850-kube-api-access-q6fqh\") pod \"93e38def-1ad5-4872-ad8c-d231cb8da850\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") "
Nov 03 23:30:49.376844 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.376813    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-metrics-certs\") pod \"93e38def-1ad5-4872-ad8c-d231cb8da850\" (UID: \"93e38def-1ad5-4872-ad8c-d231cb8da850\") "
Nov 03 23:30:49.377137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.377068    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e38def-1ad5-4872-ad8c-d231cb8da850-openshift-service-ca" pod "93e38def-1ad5-4872-ad8c-d231cb8da850" (UID: "93e38def-1ad5-4872-ad8c-d231cb8da850"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:30:49.377137 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.377085    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-plugins-config-volume" pod "beac2fcd-aaad-45eb-846b-d42c8d5ef24b" (UID: "beac2fcd-aaad-45eb-846b-d42c8d5ef24b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:30:49.377328 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.377309    5162 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-plugins-config-volume\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:49.377386 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.377334    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e38def-1ad5-4872-ad8c-d231cb8da850-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:49.379016 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.378991    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-metrics-certs" pod "93e38def-1ad5-4872-ad8c-d231cb8da850" (UID: "93e38def-1ad5-4872-ad8c-d231cb8da850"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:30:49.379119 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.379019    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-kube-api-access-x4wj6" pod "beac2fcd-aaad-45eb-846b-d42c8d5ef24b" (UID: "beac2fcd-aaad-45eb-846b-d42c8d5ef24b"). InnerVolumeSpecName "kube-api-access-x4wj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:30:49.379301 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.379283    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls" pod "93e38def-1ad5-4872-ad8c-d231cb8da850" (UID: "93e38def-1ad5-4872-ad8c-d231cb8da850"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:30:49.379368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.379309    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e38def-1ad5-4872-ad8c-d231cb8da850-kube-api-access-q6fqh" pod "93e38def-1ad5-4872-ad8c-d231cb8da850" (UID: "93e38def-1ad5-4872-ad8c-d231cb8da850"). InnerVolumeSpecName "kube-api-access-q6fqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:30:49.390524 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.390501    5162 generic.go:356] "Generic (PLEG): container finished" podID="93e38def-1ad5-4872-ad8c-d231cb8da850" containerID="1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c" exitCode=0
Nov 03 23:30:49.390618 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.390541    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"
Nov 03 23:30:49.390618 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.390579    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" event={"ID":"93e38def-1ad5-4872-ad8c-d231cb8da850","Type":"ContainerDied","Data":"1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c"}
Nov 03 23:30:49.390717 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.390636    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654" event={"ID":"93e38def-1ad5-4872-ad8c-d231cb8da850","Type":"ContainerDied","Data":"9fa58a7b0449acb26e0cf018037f26b5253ec6622537f79ac6a1f965dc00da06"}
Nov 03 23:30:49.390717 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.390666    5162 scope.go:122] "RemoveContainer" containerID="1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c"
Nov 03 23:30:49.393430 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.392069    5162 generic.go:356] "Generic (PLEG): container finished" podID="beac2fcd-aaad-45eb-846b-d42c8d5ef24b" containerID="6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d" exitCode=0
Nov 03 23:30:49.393430 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.392118    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"
Nov 03 23:30:49.393430 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.392151    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv" event={"ID":"beac2fcd-aaad-45eb-846b-d42c8d5ef24b","Type":"ContainerDied","Data":"6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d"}
Nov 03 23:30:49.393430 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.392201    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv" event={"ID":"beac2fcd-aaad-45eb-846b-d42c8d5ef24b","Type":"ContainerDied","Data":"05d82cbec6759073e457b5f3418089cddab63e6669f7f4d37ace42952df8d06a"}
Nov 03 23:30:49.400981 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.400967    5162 scope.go:122] "RemoveContainer" containerID="1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c"
Nov 03 23:30:49.401247 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:30:49.401227    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c\": container with ID starting with 1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c not found: ID does not exist" containerID="1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c"
Nov 03 23:30:49.401317 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.401256    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c"} err="failed to get container status \"1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c\": rpc error: code = NotFound desc = could not find container \"1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c\": container with ID starting with 1bb580e20de18f30a9e430b6f40460ec12fbb8e1483236ee71702f044423cc7c not found: ID does not exist"
Nov 03 23:30:49.401317 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.401279    5162 scope.go:122] "RemoveContainer" containerID="6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d"
Nov 03 23:30:49.409326 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.409311    5162 scope.go:122] "RemoveContainer" containerID="6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d"
Nov 03 23:30:49.409565 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:30:49.409546    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d\": container with ID starting with 6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d not found: ID does not exist" containerID="6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d"
Nov 03 23:30:49.409622 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.409571    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d"} err="failed to get container status \"6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d\": rpc error: code = NotFound desc = could not find container \"6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d\": container with ID starting with 6e42f84b1a3f6a6a40b7834957683a0a537f54df59fcacfef0a16d4d30b13b9d not found: ID does not exist"
Nov 03 23:30:49.478261 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.478232    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4wj6\" (UniqueName: \"kubernetes.io/projected/beac2fcd-aaad-45eb-846b-d42c8d5ef24b-kube-api-access-x4wj6\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:49.478261 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.478261    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:49.478495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.478276    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6fqh\" (UniqueName: \"kubernetes.io/projected/93e38def-1ad5-4872-ad8c-d231cb8da850-kube-api-access-q6fqh\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:49.478495 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.478290    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e38def-1ad5-4872-ad8c-d231cb8da850-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:30:49.598951 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.598872    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"]
Nov 03 23:30:49.602517 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.602489    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc1c5661-57fdd9fb8f-wd654"]
Nov 03 23:30:49.710462 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.710435    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"]
Nov 03 23:30:49.714400 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:49.714365    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv"]
Nov 03 23:30:50.709594 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:50.709561    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:50.853177 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:50.853144    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93e38def-1ad5-4872-ad8c-d231cb8da850" path="/var/lib/kubelet/pods/93e38def-1ad5-4872-ad8c-d231cb8da850/volumes"
Nov 03 23:30:50.853603 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:50.853589    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="beac2fcd-aaad-45eb-846b-d42c8d5ef24b" path="/var/lib/kubelet/pods/beac2fcd-aaad-45eb-846b-d42c8d5ef24b/volumes"
Nov 03 23:30:50.988084 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:50.988006    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:53.357072 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:53.357034    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:54.531781 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.531748    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:54.819330 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.819239    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:54.992919 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.992884    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"]
Nov 03 23:30:54.993361 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993342    5162 cpu_manager.go:414] "containerMap: removing container" podUID="44df9647-34b3-425f-b3f8-82719b9b14e0" containerName="maas-api"
Nov 03 23:30:54.993481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993363    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44df9647-34b3-425f-b3f8-82719b9b14e0" containerName="maas-api"
Nov 03 23:30:54.993481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993379    5162 cpu_manager.go:414] "containerMap: removing container" podUID="93e38def-1ad5-4872-ad8c-d231cb8da850" containerName="maas-api"
Nov 03 23:30:54.993481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993387    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93e38def-1ad5-4872-ad8c-d231cb8da850" containerName="maas-api"
Nov 03 23:30:54.993481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993465    5162 cpu_manager.go:414] "containerMap: removing container" podUID="beac2fcd-aaad-45eb-846b-d42c8d5ef24b" containerName="payload-processing"
Nov 03 23:30:54.993481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993475    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="beac2fcd-aaad-45eb-846b-d42c8d5ef24b" containerName="payload-processing"
Nov 03 23:30:54.993737 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993587    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="beac2fcd-aaad-45eb-846b-d42c8d5ef24b" containerName="payload-processing"
Nov 03 23:30:54.993737 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993610    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="44df9647-34b3-425f-b3f8-82719b9b14e0" containerName="maas-api"
Nov 03 23:30:54.993737 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.993619    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="93e38def-1ad5-4872-ad8c-d231cb8da850" containerName="maas-api"
Nov 03 23:30:54.998288 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:54.998268    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.053883 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.053842    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-05b91b-openshift-default-dockercfg-kplnw\""
Nov 03 23:30:55.102605 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.102534    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"]
Nov 03 23:30:55.131873 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.131840    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-istio-token\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.131873 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.131874    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69400e4e-9467-40b8-a591-af951c541fab-istiod-ca-cert\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.132093 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.131894    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-socket\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.132093 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.131920    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-credential-socket\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.132093 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.132006    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-envoy\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.132093 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.132082    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69400e4e-9467-40b8-a591-af951c541fab-istio-podinfo\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.132302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.132138    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m68lw\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-kube-api-access-m68lw\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.132302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.132220    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-data\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.132302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.132279    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-certs\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.136420 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.136375    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:55.233637 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233600    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-istio-token\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233637 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233632    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69400e4e-9467-40b8-a591-af951c541fab-istiod-ca-cert\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233652    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-socket\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233676    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-credential-socket\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233702    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-envoy\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233733    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69400e4e-9467-40b8-a591-af951c541fab-istio-podinfo\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233795    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m68lw\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-kube-api-access-m68lw\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233828    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-data\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.233892 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.233870    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-certs\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.234286 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.234123    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-credential-socket\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.234286 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.234168    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-socket\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.234363 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.234322    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-data\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.234363 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.234349    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-certs\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.234494 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.234474    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69400e4e-9467-40b8-a591-af951c541fab-istiod-ca-cert\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.236224 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.236194    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-envoy\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.236421 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.236385    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69400e4e-9467-40b8-a591-af951c541fab-istio-podinfo\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.294444 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.294412    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-istio-token\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.294602 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.294491    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m68lw\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-kube-api-access-m68lw\") pod \"e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") " pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.311513 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.311490    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:55.406443 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.406383    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:55.733418 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.733381    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"]
Nov 03 23:30:55.734919 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:30:55.734885    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69400e4e_9467_40b8_a591_af951c541fab.slice/crio-f3be7d0557d0615c4aaac53037a249fd15497332630c0fb409ae243091d34a9d WatchSource:0}: Error finding container f3be7d0557d0615c4aaac53037a249fd15497332630c0fb409ae243091d34a9d: Status 404 returned error can't find the container with id f3be7d0557d0615c4aaac53037a249fd15497332630c0fb409ae243091d34a9d
Nov 03 23:30:55.737051 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.737018    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:30:55.737140 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.737096    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:30:55.737203 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:55.737138    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:30:56.432747 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:56.432710    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" event={"ID":"69400e4e-9467-40b8-a591-af951c541fab","Type":"ContainerStarted","Data":"85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21"}
Nov 03 23:30:56.432945 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:56.432767    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" event={"ID":"69400e4e-9467-40b8-a591-af951c541fab","Type":"ContainerStarted","Data":"f3be7d0557d0615c4aaac53037a249fd15497332630c0fb409ae243091d34a9d"}
Nov 03 23:30:56.597794 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:56.597744    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" podStartSLOduration=2.59772938 podStartE2EDuration="2.59772938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:30:56.596833599 +0000 UTC m=+4059.861533610" watchObservedRunningTime="2026-11-03 23:30:56.59772938 +0000 UTC m=+4059.862429390"
Nov 03 23:30:57.315062 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:57.315030    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:57.319989 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:57.319965    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:57.437856 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:57.437827    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:57.438646 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:57.438626    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:30:59.464197 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:59.464147    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:30:59.667554 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:30:59.667517    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:02.685553 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:02.685519    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:02.889771 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:02.889727    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:05.497613 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.497581    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"]
Nov 03 23:31:05.501234 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.501216    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.554247 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.554214    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-05b91b-serving-cert\""
Nov 03 23:31:05.604194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.604161    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"]
Nov 03 23:31:05.647617 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.647585    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f82372-4c5f-4d25-bfef-0db393463ff6-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.647788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.647638    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.647788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.647716    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-metrics-certs\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.647788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.647777    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhlnv\" (UniqueName: \"kubernetes.io/projected/e6f82372-4c5f-4d25-bfef-0db393463ff6-kube-api-access-zhlnv\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.748941 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.748855    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhlnv\" (UniqueName: \"kubernetes.io/projected/e6f82372-4c5f-4d25-bfef-0db393463ff6-kube-api-access-zhlnv\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.748941 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.748938    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f82372-4c5f-4d25-bfef-0db393463ff6-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.749139 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.748970    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.749139 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.748993    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-metrics-certs\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.749627 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.749600    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f82372-4c5f-4d25-bfef-0db393463ff6-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.751651 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.751627    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-metrics-certs\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.751759 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.751630    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.758438 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.758404    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:05.808030 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.808002    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhlnv\" (UniqueName: \"kubernetes.io/projected/e6f82372-4c5f-4d25-bfef-0db393463ff6-kube-api-access-zhlnv\") pod \"maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.814898 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.814874    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:05.960158 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:05.960124    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:06.187262 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:06.187233    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"]
Nov 03 23:31:06.188245 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:31:06.188219    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f82372_4c5f_4d25_bfef_0db393463ff6.slice/crio-1664fe51478a8816cbaeae55525f2630cd5ed592f965c77b47a6a3e5a264bd4d WatchSource:0}: Error finding container 1664fe51478a8816cbaeae55525f2630cd5ed592f965c77b47a6a3e5a264bd4d: Status 404 returned error can't find the container with id 1664fe51478a8816cbaeae55525f2630cd5ed592f965c77b47a6a3e5a264bd4d
Nov 03 23:31:06.487149 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:06.487067    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx" event={"ID":"e6f82372-4c5f-4d25-bfef-0db393463ff6","Type":"ContainerStarted","Data":"1664fe51478a8816cbaeae55525f2630cd5ed592f965c77b47a6a3e5a264bd4d"}
Nov 03 23:31:07.494555 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:07.494521    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx" event={"ID":"e6f82372-4c5f-4d25-bfef-0db393463ff6","Type":"ContainerStarted","Data":"063f5e7c95337eed59ee94a88bab4d81db6882c9bf28beb1d3e99df5de67a51a"}
Nov 03 23:31:07.495047 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:07.494591    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:07.609531 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:07.609479    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx" podStartSLOduration=1.658414882 podStartE2EDuration="2.609465056s" totalImagesPullingTime="951.050174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:31:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:31:07.607654918 +0000 UTC m=+4070.872354930" watchObservedRunningTime="2026-11-03 23:31:07.609465056 +0000 UTC m=+4070.874165067"
Nov 03 23:31:10.082912 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:31:10.082870    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:31:10.166671 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:10.166630    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:10.711243 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:10.711198    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:13.506935 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:13.506908    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:31:15.619670 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:15.619635    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:20.407958 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:20.407882    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:23.607504 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:23.607471    5162 generic.go:356] "Generic (PLEG): container finished" podID="69400e4e-9467-40b8-a591-af951c541fab" containerID="85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21" exitCode=0
Nov 03 23:31:23.607945 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:23.607550    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" event={"ID":"69400e4e-9467-40b8-a591-af951c541fab","Type":"ContainerDied","Data":"85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21"}
Nov 03 23:31:23.607945 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:23.607939    5162 scope.go:122] "RemoveContainer" containerID="85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21"
Nov 03 23:31:23.609213 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:23.609187    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:31:23.609267 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:23.609252    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:31:23.609302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:23.609284    5162 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 03 23:31:24.201648 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:24.201613    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:24.614709 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:24.614627    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" event={"ID":"69400e4e-9467-40b8-a591-af951c541fab","Type":"ContainerStarted","Data":"e726420259696138c65257dea93da048f43af760976497642234404c385c58a1"}
Nov 03 23:31:25.314986 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:25.314952    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:31:25.316091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:25.316065    5162 patch_prober.go:29] interesting pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 03 23:31:25.316213 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:25.316118    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 03 23:31:25.522985 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:25.522952    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:26.315612 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:26.315577    5162 patch_prober.go:29] interesting pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 03 23:31:26.316038 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:26.315632    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 03 23:31:27.315120 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:27.315085    5162 patch_prober.go:29] interesting pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 03 23:31:27.315305 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:27.315142    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 03 23:31:27.438222 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:27.438180    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:31:28.314747 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:28.314709    5162 patch_prober.go:29] interesting pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 03 23:31:28.314906 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:28.314762    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 03 23:31:29.315378 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:29.315343    5162 patch_prober.go:29] interesting pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Nov 03 23:31:29.315841 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:29.315417    5162 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Nov 03 23:31:30.314441 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:30.314375    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:30.318444 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:30.318419    5162 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:31:30.319542 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:30.319524    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:31:35.124122 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:35.124084    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:39.907133 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:39.907089    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:40.083748 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:31:40.083714    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:31:44.800320 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:44.800282    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:45.507857 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:45.507819    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:49.611481 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:49.611436    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:54.431315 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:54.431271    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:55.169676 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:55.169638    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:31:59.216309 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:31:59.216267    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:04.026913 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:04.026870    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:08.828014 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:08.827973    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:10.083831 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:32:10.083798    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:32:10.515950 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:10.515909    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:13.637744 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:13.637711    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:18.513724 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:18.513682    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:23.306150 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:23.306089    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:28.113312 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:28.113246    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:32.909563 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:32.909521    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:37.715635 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:37.715547    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:40.083430 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:32:40.083380    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:32:42.382638 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:42.382604    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:44.605748 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:44.605716    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:45.862211 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:45.862167    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:47.412820 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:47.412779    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:48.397788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:48.397756    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:48.593209 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:48.593151    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:52.210418 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:52.210355    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:55.508385 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:55.508345    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:32:57.017403 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:32:57.017346    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:02.122883 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:02.122847    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:06.909776 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:06.909741    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:09.553032 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:09.550161    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:09.756798 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:09.756761    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:10.083814 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:33:10.083773    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:33:10.862724 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:10.862689    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:16.024079 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.024037    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"]
Nov 03 23:33:16.024614 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.024259    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx" podUID="e6f82372-4c5f-4d25-bfef-0db393463ff6" containerName="maas-api" containerID="cri-o://063f5e7c95337eed59ee94a88bab4d81db6882c9bf28beb1d3e99df5de67a51a" gracePeriod=30
Nov 03 23:33:16.205220 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.205182    5162 generic.go:356] "Generic (PLEG): container finished" podID="e6f82372-4c5f-4d25-bfef-0db393463ff6" containerID="063f5e7c95337eed59ee94a88bab4d81db6882c9bf28beb1d3e99df5de67a51a" exitCode=0
Nov 03 23:33:16.205421 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.205331    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx" event={"ID":"e6f82372-4c5f-4d25-bfef-0db393463ff6","Type":"ContainerDied","Data":"063f5e7c95337eed59ee94a88bab4d81db6882c9bf28beb1d3e99df5de67a51a"}
Nov 03 23:33:16.211479 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.211443    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:16.293420 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.293379    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:33:16.428893 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.428845    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-metrics-certs\") pod \"e6f82372-4c5f-4d25-bfef-0db393463ff6\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") "
Nov 03 23:33:16.429070 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.428909    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-maas-api-tls\") pod \"e6f82372-4c5f-4d25-bfef-0db393463ff6\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") "
Nov 03 23:33:16.429070 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.428955    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6f82372-4c5f-4d25-bfef-0db393463ff6-kube-api-access-zhlnv\" (UniqueName: \"kubernetes.io/projected/e6f82372-4c5f-4d25-bfef-0db393463ff6-kube-api-access-zhlnv\") pod \"e6f82372-4c5f-4d25-bfef-0db393463ff6\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") "
Nov 03 23:33:16.429194 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.429098    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6f82372-4c5f-4d25-bfef-0db393463ff6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f82372-4c5f-4d25-bfef-0db393463ff6-openshift-service-ca\") pod \"e6f82372-4c5f-4d25-bfef-0db393463ff6\" (UID: \"e6f82372-4c5f-4d25-bfef-0db393463ff6\") "
Nov 03 23:33:16.430316 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.430282    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f82372-4c5f-4d25-bfef-0db393463ff6-openshift-service-ca" pod "e6f82372-4c5f-4d25-bfef-0db393463ff6" (UID: "e6f82372-4c5f-4d25-bfef-0db393463ff6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:33:16.432881 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.432845    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f82372-4c5f-4d25-bfef-0db393463ff6-kube-api-access-zhlnv" pod "e6f82372-4c5f-4d25-bfef-0db393463ff6" (UID: "e6f82372-4c5f-4d25-bfef-0db393463ff6"). InnerVolumeSpecName "kube-api-access-zhlnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:33:16.433168 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.433140    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-maas-api-tls" pod "e6f82372-4c5f-4d25-bfef-0db393463ff6" (UID: "e6f82372-4c5f-4d25-bfef-0db393463ff6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:33:16.433302 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.433225    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-metrics-certs" pod "e6f82372-4c5f-4d25-bfef-0db393463ff6" (UID: "e6f82372-4c5f-4d25-bfef-0db393463ff6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:33:16.530838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.530744    5162 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f82372-4c5f-4d25-bfef-0db393463ff6-openshift-service-ca\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:16.530838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.530781    5162 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-metrics-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:16.530838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.530796    5162 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6f82372-4c5f-4d25-bfef-0db393463ff6-maas-api-tls\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:16.530838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:16.530805    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhlnv\" (UniqueName: \"kubernetes.io/projected/e6f82372-4c5f-4d25-bfef-0db393463ff6-kube-api-access-zhlnv\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:17.211961 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:17.211916    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx" event={"ID":"e6f82372-4c5f-4d25-bfef-0db393463ff6","Type":"ContainerDied","Data":"1664fe51478a8816cbaeae55525f2630cd5ed592f965c77b47a6a3e5a264bd4d"}
Nov 03 23:33:17.211961 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:17.211952    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"
Nov 03 23:33:17.212527 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:17.211991    5162 scope.go:122] "RemoveContainer" containerID="063f5e7c95337eed59ee94a88bab4d81db6882c9bf28beb1d3e99df5de67a51a"
Nov 03 23:33:17.369838 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:17.369795    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"]
Nov 03 23:33:17.417642 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:17.417605    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-05b91b-545f4bdffc-th5zx"]
Nov 03 23:33:17.704091 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:17.704048    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:17.978438 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:17.978338    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:18.184069 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:18.184034    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:18.739564 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:18.739530    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:18.853515 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:18.853482    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6f82372-4c5f-4d25-bfef-0db393463ff6" path="/var/lib/kubelet/pods/e6f82372-4c5f-4d25-bfef-0db393463ff6/volumes"
Nov 03 23:33:24.543702 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:24.543622    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:24.764062 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:24.764026    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:24.771519 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:24.771488    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"]
Nov 03 23:33:24.771724 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:24.771703    5162 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy" containerID="cri-o://e726420259696138c65257dea93da048f43af760976497642234404c385c58a1" gracePeriod=30
Nov 03 23:33:24.963284 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:24.963247    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:30.017331 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.017300    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:33:30.185779 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.185715    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-data\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.185779 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.185766    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-kube-api-access-m68lw\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-kube-api-access-m68lw\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.185799    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-istio-token\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-istio-token\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.185876    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-certs\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.185925    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69400e4e-9467-40b8-a591-af951c541fab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69400e4e-9467-40b8-a591-af951c541fab-istiod-ca-cert\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186043 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.185977    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-socket\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186253 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186051    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-envoy\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186253 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186051    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-data" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:33:30.186253 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186079    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/69400e4e-9467-40b8-a591-af951c541fab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69400e4e-9467-40b8-a591-af951c541fab-istio-podinfo\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186253 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186118    5162 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-credential-socket\") pod \"69400e4e-9467-40b8-a591-af951c541fab\" (UID: \"69400e4e-9467-40b8-a591-af951c541fab\") "
Nov 03 23:33:30.186487 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186276    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-certs" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:33:30.186541 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186525    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-credential-socket" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:33:30.186623 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186606    5162 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-credential-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.186693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186627    5162 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-data\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.186693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186641    5162 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-certs\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.186957 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.186934    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69400e4e-9467-40b8-a591-af951c541fab-istiod-ca-cert" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:33:30.187234 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.187131    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-socket" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:33:30.189673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.189633    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69400e4e-9467-40b8-a591-af951c541fab-istio-podinfo" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:33:30.189673 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.189658    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-kube-api-access-m68lw" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "kube-api-access-m68lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:33:30.189966 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.189933    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-istio-token" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:33:30.190064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.189997    5162 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-envoy" pod "69400e4e-9467-40b8-a591-af951c541fab" (UID: "69400e4e-9467-40b8-a591-af951c541fab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:33:30.285938 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.285901    5162 generic.go:356] "Generic (PLEG): container finished" podID="69400e4e-9467-40b8-a591-af951c541fab" containerID="e726420259696138c65257dea93da048f43af760976497642234404c385c58a1" exitCode=0
Nov 03 23:33:30.286134 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.285960    5162 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"
Nov 03 23:33:30.286134 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.285984    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" event={"ID":"69400e4e-9467-40b8-a591-af951c541fab","Type":"ContainerDied","Data":"e726420259696138c65257dea93da048f43af760976497642234404c385c58a1"}
Nov 03 23:33:30.286134 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.286048    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp" event={"ID":"69400e4e-9467-40b8-a591-af951c541fab","Type":"ContainerDied","Data":"f3be7d0557d0615c4aaac53037a249fd15497332630c0fb409ae243091d34a9d"}
Nov 03 23:33:30.286134 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.286077    5162 scope.go:122] "RemoveContainer" containerID="e726420259696138c65257dea93da048f43af760976497642234404c385c58a1"
Nov 03 23:33:30.287368 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.287347    5162 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m68lw\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-kube-api-access-m68lw\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.287447 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.287377    5162 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69400e4e-9467-40b8-a591-af951c541fab-istio-token\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.287447 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.287423    5162 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69400e4e-9467-40b8-a591-af951c541fab-istiod-ca-cert\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.287447 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.287439    5162 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-workload-socket\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.287556 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.287454    5162 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69400e4e-9467-40b8-a591-af951c541fab-istio-envoy\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.287556 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.287470    5162 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69400e4e-9467-40b8-a591-af951c541fab-istio-podinfo\") on node \"ip-10-0-143-0.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 23:33:30.298435 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.298382    5162 scope.go:122] "RemoveContainer" containerID="85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21"
Nov 03 23:33:30.307341 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.307320    5162 scope.go:122] "RemoveContainer" containerID="e726420259696138c65257dea93da048f43af760976497642234404c385c58a1"
Nov 03 23:33:30.307672 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:33:30.307651    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e726420259696138c65257dea93da048f43af760976497642234404c385c58a1\": container with ID starting with e726420259696138c65257dea93da048f43af760976497642234404c385c58a1 not found: ID does not exist" containerID="e726420259696138c65257dea93da048f43af760976497642234404c385c58a1"
Nov 03 23:33:30.307728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.307684    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e726420259696138c65257dea93da048f43af760976497642234404c385c58a1"} err="failed to get container status \"e726420259696138c65257dea93da048f43af760976497642234404c385c58a1\": rpc error: code = NotFound desc = could not find container \"e726420259696138c65257dea93da048f43af760976497642234404c385c58a1\": container with ID starting with e726420259696138c65257dea93da048f43af760976497642234404c385c58a1 not found: ID does not exist"
Nov 03 23:33:30.307728 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.307706    5162 scope.go:122] "RemoveContainer" containerID="85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21"
Nov 03 23:33:30.307960 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:33:30.307944    5162 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21\": container with ID starting with 85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21 not found: ID does not exist" containerID="85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21"
Nov 03 23:33:30.308013 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.307964    5162 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21"} err="failed to get container status \"85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21\": rpc error: code = NotFound desc = could not find container \"85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21\": container with ID starting with 85a03afc3a716f88457cb9b1e60c0692e602d9735906c35fadd3f090342b6f21 not found: ID does not exist"
Nov 03 23:33:30.445411 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.445292    5162 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"]
Nov 03 23:33:30.452450 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.452409    5162 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp"]
Nov 03 23:33:30.855342 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:30.855244    5162 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69400e4e-9467-40b8-a591-af951c541fab" path="/var/lib/kubelet/pods/69400e4e-9467-40b8-a591-af951c541fab/volumes"
Nov 03 23:33:40.084951 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:33:40.084908    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:33:55.856683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:55.856639    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:33:56.216476 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:33:56.216432    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:34:10.085093 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:34:10.085010    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:34:11.244943 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:34:11.244907    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:34:40.084142 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:34:40.084103    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:34:44.195416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:34:44.192482    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:34:44.195416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:34:44.192576    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6" podUID="4ddece53-145f-4433-b5ad-8aa6b6576873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:34:44.263416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:34:44.262504    5162 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:34:44.263416 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:34:44.262575    5162 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28" podUID="fc872ba5-e18e-4687-81f9-bcbb760f8faa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:34:56.289676 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:34:56.289636    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:34:56.585570 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:34:56.585476    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:35:02.369948 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:02.369904    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:35:04.458877 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:04.458844    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-hqw4s_c22f1c68-cb3b-49d3-bda7-b70a4aba7265/proxy/0.log"
Nov 03 23:35:05.582753 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:05.582723    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5b4b4b5f89-pz627_98136380-67c7-4701-8cea-6b44a8ff4802/maas-api/0.log"
Nov 03 23:35:10.085049 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:35:10.085007    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:35:11.173853 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:11.173815    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-76f56c5945rhvx4_ce5c0c49-89bd-4f97-87ab-2d33724b913c/istio-proxy/0.log"
Nov 03 23:35:11.599225 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:11.599134    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:35:11.791982 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:11.791954    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84cff947f5-5xh27_377946e7-e808-40db-aebc-8714cd88b181/kube-auth-proxy/0.log"
Nov 03 23:35:12.238811 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:12.238781    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-59f957fdd7-66p28_fc872ba5-e18e-4687-81f9-bcbb760f8faa/istio-proxy/0.log"
Nov 03 23:35:12.707517 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:12.707486    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-59f957fdd7-chpk6_4ddece53-145f-4433-b5ad-8aa6b6576873/istio-proxy/0.log"
Nov 03 23:35:13.780208 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:13.780172    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-59f957fdd7-p59kr_e2eb0dd5-b5d0-40b6-a00b-7761d4adadda/istio-proxy/0.log"
Nov 03 23:35:14.795426 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:14.795383    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5_9227810a-8c4b-41e8-a17b-1add6a0428e1/istio-proxy/0.log"
Nov 03 23:35:26.746071 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:26.746037    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-4lpz9_26e27a85-625a-45c5-aba9-d60be04671b7/main/0.log"
Nov 03 23:35:27.257670 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:27.257636    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-bjwpl_c4cff4d6-fdcd-4171-956e-046ae097bca5/main/0.log"
Nov 03 23:35:27.964507 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:27.964475    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2_4def062f-125a-468b-a494-9aa4d4f99ef7/main/0.log"
Nov 03 23:35:28.196821 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:28.196787    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp_9a63990d-9392-4c4a-b804-3dfa65e4a824/main/0.log"
Nov 03 23:35:40.084606 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:35:40.084526    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:35:56.564153 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:35:56.564106    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:36:10.083729 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:36:10.083685    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:36:11.947541 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:11.947498    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:36:20.532316 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:20.532277    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-hqw4s_c22f1c68-cb3b-49d3-bda7-b70a4aba7265/proxy/0.log"
Nov 03 23:36:21.477563 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:21.477530    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5b4b4b5f89-pz627_98136380-67c7-4701-8cea-6b44a8ff4802/maas-api/0.log"
Nov 03 23:36:22.033029 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:22.032989    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:36:26.203172 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:26.203140    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-76f56c5945rhvx4_ce5c0c49-89bd-4f97-87ab-2d33724b913c/istio-proxy/0.log"
Nov 03 23:36:26.767232 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:26.767202    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84cff947f5-5xh27_377946e7-e808-40db-aebc-8714cd88b181/kube-auth-proxy/0.log"
Nov 03 23:36:27.209818 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:27.209782    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-59f957fdd7-66p28_fc872ba5-e18e-4687-81f9-bcbb760f8faa/istio-proxy/0.log"
Nov 03 23:36:27.645865 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:27.645784    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-59f957fdd7-chpk6_4ddece53-145f-4433-b5ad-8aa6b6576873/istio-proxy/0.log"
Nov 03 23:36:28.668373 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:28.668338    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-59f957fdd7-p59kr_e2eb0dd5-b5d0-40b6-a00b-7761d4adadda/istio-proxy/0.log"
Nov 03 23:36:29.604530 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:29.604489    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5_9227810a-8c4b-41e8-a17b-1add6a0428e1/istio-proxy/0.log"
Nov 03 23:36:31.230474 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:31.230447    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-4lpz9_26e27a85-625a-45c5-aba9-d60be04671b7/main/0.log"
Nov 03 23:36:31.737061 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:31.737031    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-bjwpl_c4cff4d6-fdcd-4171-956e-046ae097bca5/main/0.log"
Nov 03 23:36:32.438918 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:32.438883    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-kngc2_4def062f-125a-468b-a494-9aa4d4f99ef7/main/0.log"
Nov 03 23:36:32.678059 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:32.678031    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-tz7pp_9a63990d-9392-4c4a-b804-3dfa65e4a824/main/0.log"
Nov 03 23:36:40.085613 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:36:40.085566    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:36:56.911260 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:36:56.911222    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:37:00.736769 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:37:00.736733    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:37:10.085406 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:37:10.085306    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:37:12.279626 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:37:12.279589    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:37:25.416668 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:37:25.416622    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:37:25.619491 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:37:25.619451    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:37:40.084581 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:37:40.084539    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:37:57.262343 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:37:57.262306    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:37:59.557136 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:37:59.557104    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:38:10.083666 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:38:10.083628    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:38:12.613364 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:12.613328    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:38:18.619674 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.619632    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-md998/must-gather-rhc4n"]
Nov 03 23:38:18.620064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620035    5162 cpu_manager.go:414] "containerMap: removing container" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy"
Nov 03 23:38:18.620064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620046    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy"
Nov 03 23:38:18.620142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620072    5162 cpu_manager.go:414] "containerMap: removing container" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy"
Nov 03 23:38:18.620142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620077    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy"
Nov 03 23:38:18.620142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620087    5162 cpu_manager.go:414] "containerMap: removing container" podUID="e6f82372-4c5f-4d25-bfef-0db393463ff6" containerName="maas-api"
Nov 03 23:38:18.620142 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620094    5162 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6f82372-4c5f-4d25-bfef-0db393463ff6" containerName="maas-api"
Nov 03 23:38:18.620267 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620206    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy"
Nov 03 23:38:18.620267 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620229    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6f82372-4c5f-4d25-bfef-0db393463ff6" containerName="maas-api"
Nov 03 23:38:18.620439 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.620424    5162 memory_manager.go:365] "RemoveStaleState removing state" podUID="69400e4e-9467-40b8-a591-af951c541fab" containerName="istio-proxy"
Nov 03 23:38:18.623683 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.623668    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.674000 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.673970    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-md998\"/\"openshift-service-ca.crt\""
Nov 03 23:38:18.684606 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.684571    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmlh5\" (UniqueName: \"kubernetes.io/projected/885b84ca-0472-44d7-9dd9-69ee4cc726f1-kube-api-access-jmlh5\") pod \"must-gather-rhc4n\" (UID: \"885b84ca-0472-44d7-9dd9-69ee4cc726f1\") " pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.684788 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.684760    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/885b84ca-0472-44d7-9dd9-69ee4cc726f1-must-gather-output\") pod \"must-gather-rhc4n\" (UID: \"885b84ca-0472-44d7-9dd9-69ee4cc726f1\") " pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.692881 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.692849    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:38:18.721653 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.721621    5162 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-md998\"/\"kube-root-ca.crt\""
Nov 03 23:38:18.722282 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.722262    5162 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-md998\"/\"default-dockercfg-sg8mv\""
Nov 03 23:38:18.724051 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.724031    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-md998/must-gather-rhc4n"]
Nov 03 23:38:18.786119 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.786084    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/885b84ca-0472-44d7-9dd9-69ee4cc726f1-must-gather-output\") pod \"must-gather-rhc4n\" (UID: \"885b84ca-0472-44d7-9dd9-69ee4cc726f1\") " pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.786324 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.786134    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmlh5\" (UniqueName: \"kubernetes.io/projected/885b84ca-0472-44d7-9dd9-69ee4cc726f1-kube-api-access-jmlh5\") pod \"must-gather-rhc4n\" (UID: \"885b84ca-0472-44d7-9dd9-69ee4cc726f1\") " pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.786541 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.786519    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/885b84ca-0472-44d7-9dd9-69ee4cc726f1-must-gather-output\") pod \"must-gather-rhc4n\" (UID: \"885b84ca-0472-44d7-9dd9-69ee4cc726f1\") " pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.844786 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.844752    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmlh5\" (UniqueName: \"kubernetes.io/projected/885b84ca-0472-44d7-9dd9-69ee4cc726f1-kube-api-access-jmlh5\") pod \"must-gather-rhc4n\" (UID: \"885b84ca-0472-44d7-9dd9-69ee4cc726f1\") " pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.937642 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.937607    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-md998/must-gather-rhc4n"
Nov 03 23:38:18.945490 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:18.945461    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:38:19.318919 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:19.318886    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-md998/must-gather-rhc4n"]
Nov 03 23:38:19.320103 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:38:19.320072    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod885b84ca_0472_44d7_9dd9_69ee4cc726f1.slice/crio-53e0232130c9871dfa0225c8606d0b65bd5c2f221474c1e7bb195a852d938a75 WatchSource:0}: Error finding container 53e0232130c9871dfa0225c8606d0b65bd5c2f221474c1e7bb195a852d938a75: Status 404 returned error can't find the container with id 53e0232130c9871dfa0225c8606d0b65bd5c2f221474c1e7bb195a852d938a75
Nov 03 23:38:19.322121 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:19.322104    5162 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 23:38:19.861374 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:19.861340    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-md998/must-gather-rhc4n" event={"ID":"885b84ca-0472-44d7-9dd9-69ee4cc726f1","Type":"ContainerStarted","Data":"53e0232130c9871dfa0225c8606d0b65bd5c2f221474c1e7bb195a852d938a75"}
Nov 03 23:38:21.875144 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:21.875108    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-md998/must-gather-rhc4n" event={"ID":"885b84ca-0472-44d7-9dd9-69ee4cc726f1","Type":"ContainerStarted","Data":"9b87e827e143085051bdba381583dcf9b59e2edc3396568e99613841751ea679"}
Nov 03 23:38:21.875595 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:21.875154    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-md998/must-gather-rhc4n" event={"ID":"885b84ca-0472-44d7-9dd9-69ee4cc726f1","Type":"ContainerStarted","Data":"480c4eecea9f0d233cd69e4918aadeb2fd6a9dffdc3bfd4aef09d708fd655e00"}
Nov 03 23:38:21.987014 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:21.986945    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-md998/must-gather-rhc4n" podStartSLOduration=2.50131636 podStartE2EDuration="3.986928877s" totalImagesPullingTime="1.485612517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:38:21.986665458 +0000 UTC m=+4505.251365469" watchObservedRunningTime="2026-11-03 23:38:21.986928877 +0000 UTC m=+4505.251628889"
Nov 03 23:38:24.370606 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:24.370566    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:38:24.575651 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:24.575616    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:38:27.350741 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:27.350711    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-86xxh_48ed2fb2-b25c-48fd-84c3-59a9a1ec4d44/global-pull-secret-syncer/0.log"
Nov 03 23:38:30.389418 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:30.388992    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log"
Nov 03 23:38:32.533675 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:32.533642    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-0.us-west-2.compute.internal_ffed3ef5033ca77208390bb2ae5eac19/haproxy/0.log"
Nov 03 23:38:37.888172 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:37.888084    5162 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"]
Nov 03 23:38:37.895133 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:37.895105    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:37.946006 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:37.945950    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"]
Nov 03 23:38:38.033064 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.033036    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k2s2\" (UniqueName: \"kubernetes.io/projected/2b31511d-a4e7-41b5-a7e7-055010db07d9-kube-api-access-9k2s2\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.033237 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.033096    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-sys\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.033237 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.033151    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-podres\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.033237 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.033193    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-proc\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.033350 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.033317    5162 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-lib-modules\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.134847 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.134806    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-sys\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135047 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.134856    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-podres\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135047 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.134941    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-sys\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135047 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.134944    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-proc\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135047 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.135005    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-proc\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135047 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.135010    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-podres\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135283 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.135086    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-lib-modules\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135283 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.135148    5162 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k2s2\" (UniqueName: \"kubernetes.io/projected/2b31511d-a4e7-41b5-a7e7-055010db07d9-kube-api-access-9k2s2\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.135283 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.135209    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b31511d-a4e7-41b5-a7e7-055010db07d9-lib-modules\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.194678 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.194649    5162 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k2s2\" (UniqueName: \"kubernetes.io/projected/2b31511d-a4e7-41b5-a7e7-055010db07d9-kube-api-access-9k2s2\") pod \"perf-node-gather-daemonset-wtkz9\" (UID: \"2b31511d-a4e7-41b5-a7e7-055010db07d9\") " pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.207644 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.207619    5162 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:38.583803 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:38.583767    5162 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"]
Nov 03 23:38:38.585542 ip-10-0-143-0 kubenswrapper[5162]: W1103 23:38:38.585510    5162 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2b31511d_a4e7_41b5_a7e7_055010db07d9.slice/crio-6cb32a17ec46b7e3694ae4fdace2131b55e6bd10f785bd95cc9463beddc40719 WatchSource:0}: Error finding container 6cb32a17ec46b7e3694ae4fdace2131b55e6bd10f785bd95cc9463beddc40719: Status 404 returned error can't find the container with id 6cb32a17ec46b7e3694ae4fdace2131b55e6bd10f785bd95cc9463beddc40719
Nov 03 23:38:39.035560 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:39.035524    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9" event={"ID":"2b31511d-a4e7-41b5-a7e7-055010db07d9","Type":"ContainerStarted","Data":"3eeb5d2a5e98b7e2ab35207e1d7d94fd56067b1ef274d9fb297ade64d16614c8"}
Nov 03 23:38:39.035983 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:39.035579    5162 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9" event={"ID":"2b31511d-a4e7-41b5-a7e7-055010db07d9","Type":"ContainerStarted","Data":"6cb32a17ec46b7e3694ae4fdace2131b55e6bd10f785bd95cc9463beddc40719"}
Nov 03 23:38:39.035983 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:39.035717    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:39.148693 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:39.148639    5162 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9" podStartSLOduration=2.148623915 podStartE2EDuration="2.148623915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 23:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:38:39.146549377 +0000 UTC m=+4522.411249389" watchObservedRunningTime="2026-11-03 23:38:39.148623915 +0000 UTC m=+4522.413323925"
Nov 03 23:38:40.083909 ip-10-0-143-0 kubenswrapper[5162]: E1103 23:38:40.083875    5162 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-d5h8h" containerName="extractor"
Nov 03 23:38:45.053345 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:45.053315    5162 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-md998/perf-node-gather-daemonset-wtkz9"
Nov 03 23:38:47.499727 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:47.499687    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5bbc88ffc8-7mgdn_16d2011b-252e-48d6-be1e-a1a9a8537b39/metrics-server/0.log"
Nov 03 23:38:47.926698 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:47.926672    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-pdkbc_a9ae4421-28f8-420f-ade1-c91aafe026ab/monitoring-plugin/0.log"
Nov 03 23:38:49.625351 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:49.625312    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jfrn4_ec301699-cf86-453b-872b-b874de2dcf0d/node-exporter/0.log"
Nov 03 23:38:50.040231 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:50.040201    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jfrn4_ec301699-cf86-453b-872b-b874de2dcf0d/kube-rbac-proxy/0.log"
Nov 03 23:38:50.455435 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:50.455404    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jfrn4_ec301699-cf86-453b-872b-b874de2dcf0d/init-textfile/0.log"
Nov 03 23:38:52.104978 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:52.104947    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rs9pf_8086e88f-4770-416e-93fd-cf324fae5917/kube-rbac-proxy-main/0.log"
Nov 03 23:38:52.521040 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:52.521010    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rs9pf_8086e88f-4770-416e-93fd-cf324fae5917/kube-rbac-proxy-self/0.log"
Nov 03 23:38:52.938173 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:52.938145    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-rs9pf_8086e88f-4770-416e-93fd-cf324fae5917/openshift-state-metrics/0.log"
Nov 03 23:38:53.371292 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:53.371208    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b6f987d-e21b-4888-b9a6-6320d52d7b85/prometheus/0.log"
Nov 03 23:38:53.794472 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:53.794434    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b6f987d-e21b-4888-b9a6-6320d52d7b85/config-reloader/0.log"
Nov 03 23:38:54.209556 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:54.209526    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b6f987d-e21b-4888-b9a6-6320d52d7b85/thanos-sidecar/0.log"
Nov 03 23:38:54.623965 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:54.623894    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b6f987d-e21b-4888-b9a6-6320d52d7b85/kube-rbac-proxy-web/0.log"
Nov 03 23:38:55.040360 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:55.040326    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b6f987d-e21b-4888-b9a6-6320d52d7b85/kube-rbac-proxy/0.log"
Nov 03 23:38:55.456290 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:55.456261    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b6f987d-e21b-4888-b9a6-6320d52d7b85/kube-rbac-proxy-thanos/0.log"
Nov 03 23:38:55.872494 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:55.872421    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b6f987d-e21b-4888-b9a6-6320d52d7b85/init-config-reloader/0.log"
Nov 03 23:38:57.568514 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:57.568478    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74dbbb49b8-2nl5k_af060dc0-cbf7-4b75-b3ff-167f74b11ba0/telemeter-client/0.log"
Nov 03 23:38:57.600661 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:57.600632    5162 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d46vm_f2c344bf-cb2b-4eef-aaa5-c7ae74e8a982/konnectivity-agent/0.log" limit=1024
Nov 03 23:38:57.984248 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:57.984221    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74dbbb49b8-2nl5k_af060dc0-cbf7-4b75-b3ff-167f74b11ba0/reload/0.log"
Nov 03 23:38:58.401488 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:58.401414    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74dbbb49b8-2nl5k_af060dc0-cbf7-4b75-b3ff-167f74b11ba0/kube-rbac-proxy/0.log"
Nov 03 23:38:58.824065 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:58.824036    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cc74c986b-hzzr2_1695e3d3-d78a-41f2-bbad-67127b64a8fe/thanos-query/0.log"
Nov 03 23:38:59.255946 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:59.255917    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cc74c986b-hzzr2_1695e3d3-d78a-41f2-bbad-67127b64a8fe/kube-rbac-proxy-web/0.log"
Nov 03 23:38:59.671036 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:38:59.671008    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cc74c986b-hzzr2_1695e3d3-d78a-41f2-bbad-67127b64a8fe/kube-rbac-proxy/0.log"
Nov 03 23:39:00.088662 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:39:00.088591    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cc74c986b-hzzr2_1695e3d3-d78a-41f2-bbad-67127b64a8fe/prom-label-proxy/0.log"
Nov 03 23:39:00.504864 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:39:00.504830    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cc74c986b-hzzr2_1695e3d3-d78a-41f2-bbad-67127b64a8fe/kube-rbac-proxy-rules/0.log"
Nov 03 23:39:00.925576 ip-10-0-143-0 kubenswrapper[5162]: I1103 23:39:00.925501    5162 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cc74c986b-hzzr2_1695e3d3-d78a-41f2-bbad-67127b64a8fe/kube-rbac-proxy-metrics/0.log"